LEGO Brick Sorter

Course: Measurements 2 (MEMS 1043) — University of Pittsburgh

Team 14: James Puzon, Kalinda Wagner, Kyleigh Motley

View on GitHub

Experimental set-up: ultrasonic distance sensors and webcam for LEGO block identification

Project Overview

This project developed a system to identify LEGO blocks based on geometric and visual data, with the potential application of sorting LEGO bricks automatically. The system combines two independent sensing approaches:

• Ultrasonic sensors + Arduino Uno: Measure block geometry (length, width, height) using non-contact distance measurement
• Web camera + Python: Capture images and extract color information using HSV-based color detection

Error analysis was performed for each measurement method to assess the capabilities and limitations of the system.

Sensor Selections

• All three distance sensors were interfaced via a single Arduino Uno for power and data acquisition
• The web camera was combined with a custom Python script for color identification

Equipment List

Color Recognition

Calibration

Before detection, the system is calibrated to build a dictionary of color ranges for accurate color classification. This calibration process is run once (or whenever new colors need to be added):

1. Center a LEGO block within the defined detection region on screen
2. The script samples and outputs HSV pixel data from the detection box
3. The user inputs the corresponding color label
4. The script appends or replaces the HSV range and label in the color database

Detection Process

A custom Python script identifies each LEGO block's color in real time using the Logitech C920s webcam:

1. LEGO piece is placed within the camera's field of view
2. Script converts the image from BGR to HSV color space
3. Measured HSV values are compared to the saved color dictionary
4. The detected color label is output to the user

The script supports both vibrant colors (red, yellow, green, blue) and neutral colors (black, white, gray, brown). A secondary window displays the full color dictionary with HSV ranges, updated in real time. Tolerance values used: Hue ±3, Saturation ±15, Value ±3.

Distance Measurements

Calibration

Sensors are calibrated using an Arduino sketch run while the measurement area is empty of LEGOs:

1. Sensors collect data for 5 seconds in each direction (X, Y, Z)
2. Data is averaged to produce the nominal distance to the wall in each direction
3. These nominal values are entered into the measurement script as reference baselines

Measurement Process

Ultrasonic sensors determine LEGO geometry by emitting and receiving sound pulses:

1. LEGO block is placed within the sensing range of all three sensors
2. Sensors fire a few milliseconds apart to avoid signal interference between them
3. For 10 seconds, each sensor records its distance to the LEGO block
4. Measured distances are subtracted from the nominal (empty-area) distance
5. Final dimensions are obtained by averaging the X, Y, and Z measurements
6. Final dimensions are recorded in Excel for analysis

Experimental Results

A sample of five LEGO blocks was measured using the test and calibration procedures. Percent and RMS errors were calculated for each block.

In general, average absolute geometric error was high, indicating issues with sensor placement, set-up design, or signal interference — particularly for height measurements. Color detection accuracy was 100%, demonstrating that the Python/OpenCV-based approach is highly reliable for this task.

Design of Experiments

Research Question: How does lighting, background color, and camera quality affect how the camera reads the LEGO brick color?

• Lighting: Dim (300 lumens) vs. Bright (1,500–2,000 lumens)
• Background Color: White vs. Black
• Camera Quality: Low (320×240) vs. High (1920×1080)

Experimental Strengths

• Simple, Low-Cost Set-Up: Uses readily available components (ultrasonic sensors, Arduino, webcam) that are easy to assemble and modify
• Non-Contact Measurements: Ultrasonic distance sensors measure geometry without touching the block, reducing wear and alignment issues
• Easy Calibration: The color recognition script can easily be updated to include new LEGO colors; distance calibration uses simple offsets
• Modular Architecture: Color detection and geometry measurement operate independently using different sensors, so components can be replaced without impacting the rest of the system

Experimental Limitations

• Geometric Constraints: System is optimized for rectangular or square blocks; irregular shapes may cause incorrect geometry classification
• Color Detection Sensitivity: Webcam performance depends heavily on lighting conditions; neutral colors (black, white, gray) are difficult to distinguish reliably under poor lighting
• Possible Sensor Interference: Ultrasonic sensors can interfere with each other when placed close together; echoes bouncing off walls may cause inaccurate readings even with small firing delays
• Surface Material and Orientation Effects: Distance sensors perform poorly with angled or irregular surfaces; LEGO studs on top surfaces can introduce height measurement errors

Future Improvements

• Additional Classification Parameters: Incorporate weight, number of studs, or surface texture to improve geometric classification
• Improved Lighting: Add dedicated LED lighting around the camera to reduce color recognition errors under variable ambient conditions
• Automated Testing: LEGO bricks are currently placed by hand; automation (e.g., a feed ramp) would improve throughput and repeatability
• More Robust Geometric Sensing: Replace ultrasonic sensors with infrared or laser distance sensors for higher accuracy at short ranges
• Sensor Filtering: Apply digital filters (e.g., moving average, Kalman) to reduce noise and improve geometric measurement accuracy
• Machine Learning Classification: Replace HSV thresholding with a trained classifier for more robust performance across lighting conditions and brick colors

Technologies Used

• Python / OpenCV: Real-time color detection using HSV color space, calibration workflow, and dictionary-based classification
• Arduino Uno: Data acquisition and timing control for three ultrasonic distance sensors
• Ultrasonic Distance Sensors (Adafruit 4007, HC-SR04): Non-contact geometry measurement
• Logitech C920s HD Webcam: High-resolution image capture for color identification
• Excel: Data recording, averaging, and error analysis