Give Me a Hand

Dean Zadok, a Ph.D. student in the Henry and Marilyn Taub Faculty of Computer Science at the Technion – Israel Institute of Technology, has developed a robotic system that could change the lives of amputees for the better. The system already allows users to play the piano and type on a computer, and Dean plans to further enhance its capabilities. “Many daily tasks, such as typing, making coffee, or changing a light bulb, are very challenging for people with amputated hands,” he explains. “Hence, the immense importance of efficient and user-friendly solutions.”

Dean Zadok presenting the development on Faculty Research Day

One of the complex challenges in robotic prosthetic arms is the ability to perform more intricate actions beyond gripping and moving objects. Existing prostheses may allow users, at best, to hold a cup and drink from it, but typing on a computer or playing the piano requires a high level of dexterity. From this complex challenge, a sophisticated project emerged, guided by Dean and three Technion faculty members: Professor Alon Wolf, a robotics and biomechanics expert from the Faculty of Mechanical Engineering, and Taub Faculty of Computer Science researchers Professor Alex Bronstein (computational learning) and Dr. Oren Salzman (robotics).

The new technology is an ultrasound-based robotic system that reads muscle movements, enabling precise and sensitive hand actions, including fine finger movements. The ultrasound system is attached to the forearm, interpreting the user’s intentions based on muscle movements, including complex and fine gestures. The research team estimates this is a significant leap that will substantially advance the field of prosthetics, providing many users with an improved quality of life.

Robotic hand

Dean Zadok, born in Tel Aviv, served in a technological role in the army before beginning his studies at the Technion. During his undergraduate studies, he joined the “Technion Formula” team, which develops race cars representing the Technion in the Formula SAE championships. Alongside fellow students Amir Biran and Tom Hirshberg, he developed a “Formula Simulator” to simulate the autonomous Formula car’s driving for testing purposes. The algorithm developed by the team allows the autonomous car to seamlessly drive in an unseen racing track, considering various factors such as weather, track conditions, speed, and more. The system was developed under the guidance of Technion Visiting Professor Kira Radinsky and Ashish Kapoor from Microsoft.

After completing his undergraduate degree, Dean went to Seattle to intern in Microsoft’s research labs. Upon his return to Israel, he began his studies for a master’s degree at the Technion. He volunteered and gained extensive experience in the field of prosthetics in Prof. Wolf’s lab and at Haifa3D, a non-profit organization aimed at creating awareness to and acquiring skills in 3D digital creation of physical object using 3D printing. He also visited the Massachusetts Institute of Technology (MIT), where he learned about the next generation of robotic prosthetics. “Participating in the Formula car development was an amazing experience,” he says, “but I always wanted to apply my knowledge for the benefit of human health. Algorithms find their application in a variety of fields, and I am glad I could harness it for the important topic of improving prosthetics for those who have suffered.”

The field of robotic prosthetics is making technological advances, but most smart prostheses currently rely on sensor stickers attached to the skin to interpret muscle signals. According to Dean, “this technology is very limiting, and what we are proposing is a new approach based on ultrasound, providing real-time dynamic information about relevant muscle movements for hand and finger motions.”

Robotic hand

Initially, the team worked on a solution to allow users to separate the attached fingers, but they soon realized there was greater potential: a continuous reconstruction of finger movements based on ultrasound tracking of the contraction muscles in the lower arm. This led to the development of a system that allows users to type on a computer and play the piano.

Dean achieved this feat during his master’s thesis and decided to continue with the project through his doctoral studies. Last summer, he was invited to London to present the project at the Institute of Electrical and Electronics Engineers (IEEE) International Conference on Robotics and Automation (ICRA).

While his achievements are already impressive, Dean still has many goals for the future. One such objective is to develop an algorithm that will not only work after training with a specific user but will also adapt well to new users. “Many people who have lost a hand give up on the prosthesis after a short period because it is heavy, cumbersome, and its effectiveness is very limited,” he said. “We are trying to develop lightweight, comfortable, and efficient solutions. To achieve this, we need to continue improving the accuracy and strength of the movements.”

The research is supported by the European Research Council (ERC), the Ministry of Science and Technology, the Israel-U.S. Binational Science Foundation (BSF), the David and Lucile Packard Foundation, and the Wein Family Foundation. The researchers thank Haifa3D for its support.

Read the full article here: https://ieeexplore.ieee.org/document/10160601