Giving Students a new learning experience
Its a on going project
An academic project where we were exploring machine learning. The theme of the our project was delayed development in students.
We explored the potential of using alternative approaches or tools to help children with intellectual disabilities learn languages. We found that digital and multisensory resources can be effective in providing these children with more accessible ways to engage with alphabets and language learning.
Timeframe Sep 2022-
Role : Ux strategy
Platform: Tangible Interface
Our Approach
The project aims to explore innovative solutions that can potentially empower these children, promote their engagement, and enhance their language learning outcomes.
Our idea works on the basis of giving children a more tactile medium to learn languages. Using their own physical bodies means they feel more connected with what they are doing and this results in them having better information retention regarding the same.
In our exploration of Machine Learning, we wanted to build something that helped children learn using this technology. Our project had two main components — The Sandbox and the “Code”
Understanding delayed development
The research on cognitive development in children with intellectual disability (ID). ID is a condition characterized by significant limitations in intellectual functioning and adaptive behavior, which covers a range of everyday social and practical skills.
The research on specific cognitive skills in children with ID. These skills include attention, working memory, executive function, language, and social cognition. The research shows that children with ID have significant difficulties in all of these areas. However, the severity of the difficulties varies depending on the specific skill and the child's overall level of intellectual functioning.
Here are some of the key findings:
Children with ID have significant delays in cognitive development.
These delays are evident in a range of cognitive skills, including attention, working memory, executive function, language, and social cognition.
The severity of the delays varies depending on the specific skill and the child's overall level of intellectual functioning.
The Sandbox
We built a sandbox that mimics the boxes used by sand artists to provide a glow underneath the sand. This was done to give the computer vision model a fixed background so that the only variable it has to deal with is what is written in the sand.
The following materials were used to create the box:
5mm MDF Sheets
3mm Acrylic Sheets
White LED Strips
Webcam
Battery/Power Supply
Tripod
Traditional Indian Coloured Sand (Rangoli)
The box was laser cut out of the mentioned material sheets, and a hole was drilled in the box to create a spot for the LEDs and Battery to be mounted.
The box was designed to be highly modular, allowing us to easily switch out the acrylic sheet above as well as the lighting system inside
The Code
It would be a disservice to the coding community to call this section “coding”, but for the lack of a better term, the code was prepared on the code blocks software — Mblock. Mblock has an inbuilt Cognitive Services extension which allowed us to natively train a supervised classification-based model and teach it the characters of the alphabet.
The code was designed in a way that when the ML Model recognizes a specific image, the code outputs a message that is read by all the other sprites in the code, and the appropriate code is displayed.
Final Output
Our Further Steps and Observations
The system needs to be completely integrated in order to be fit for testing or distribution.
In the current examples, we have used traditional colored sand — Rangoli as our sand. This was done due to it being widely available as well as the high contrast it provided for the computer vision to easily recognize text. However, this is not optimal for testing or further use, as the Rangoli sand stains the user’s hands due to its powdered dye. Further, its toxicity is high enough for it to be a risk if a member of our target audience, a child, would accidently put the sand in their mouth. To address this, the next version of this prototype will use Dyed Bombay Rava instead of the Rangoli. This will address both of our issues, as Bombay rava is a food, and if dyed with food safe dye, it will be completely edible. It is however, majorly tasteless, so users will not want to eat it, even though they would be safe if they did.
The code is currently optimized for a presentation setup, it will need to be redesigned for testing and distribution.
Check These Out Next
Case Study
Metro Delhi International School
UX research | Ui designs | Figma
View Project
Publishing
My Remains
Graphic Design | Book cover | Illustrations
View Project
