Guesture control screen reading using face detection The increasing development of interfacing options to microprocessors have led to numerous applications scaling from general daily life applications (such as elevator, traffic light controller...etc) to every other activity using IoT technologies. In this paper we propose a mechanism of interfacing Arduino with ultrasonic sensor used to magnify the screen according to the distance of face from the screen. This project comes in handy for comfortable reading for people with low vision and also helpful for normal vision people for better reading of content on the screen. Instead of using keyboard, mouse or joystick, we can use our hand gestures to control certain functions other than above such as play/pause video, move to next/ previous slide while presenting, scroll up/down pdf page. We use hand gestures to control such applications, where number of gestures can be increased using more number of ultrasonic sensors. So the main idea behind the project is by using already present on laptop camera to detect the face and adjust the screen magnification based on face position by placing ultrasonic sensor on the top of the laptop screen. Calculation of distance from the screen is done by ultrasonic sensor. Counting on the information from Arduino that is send to the python through the serial port, this information will be then read by python which is running on the computer in order to perform certain actions. We mainly focused on magnification adjustment below here whereas also explained the gestures part with two ultrasonic sensors included.
Arduino Uno: The Arduino Uno is an open source microcontroller board based on the ATMega 328P microcontroller developed by Arduino.cc. The board is equipped with sets of digital and analog input/output (I/O) pins that may be interfaced to various expansion boards (shields) and other circuits. It has 20 digital input/output pins (of which 6 can be used as PWM outputs and 6 can be used as analog inputs), a 16 MHz resonator, a USB connection, a power jack, an in-circuit system programming (ICSP) header, and a reset button. It contains everything needed to support the microcontroller; simply connect it to a computer (or appropriate wall power adapter) with a USB cable or power it with a AC-to-DC adapter or battery to get started. The Uno differs from all preceding boards in that it does not use the FTDI USB-to-serial driver chip. Instead, it features an ATmega16U2 programmed as a USB-to-serial converter. This auxiliary microcontroller has its own USB bootloader, which allows advanced users to reprogram it. General Pin functions: • LED: There is a built-in LED driven by digital pin 13. When the pin is HIGH value, the LED is on, when the pin is LOW, it's off. • VIN: The input voltage to the Arduino/Genuino board when it's using an external power source (as opposed to 5 volts from the USB connection or other regulated power source). You can supply voltage through this pin, or, if supplying voltage via the power jack, access it through this pin. • 5V: This pin outputs a regulated 5V from the regulator on the board. The board can be supplied with power either from the DC power jack (7 - 20V), the USB connector (5V), or the VIN pin of the board (7-20V). Supplying voltage via the 5V or 3.3V pins bypasses the regulator, and can damage the board. • 3V3: A 3.3 volt supply generated by the on-board regulator. Maximum current draw is 50 mA. • GND: Ground pins. • Reset: Typically used to add a reset button to shields which block the one on the board.
ULTRASONIC SENSOR: HC SR04 is an ultrasonic sensor, also known as an ultrasonic transducer that is based on a transmitter and receiver and mainly used to determine the distance from the target object. The amount of time it takes to send and receive waves will determine how far the object is placed from the sensor. It mainly depends on the sound waves working on “non-contact” technology. The required distance of the target object is measured without any damage, giving you accurate and precise details. This sensor comes with a range between 2cm to 400cm and is used in a wide range of applications including speed and direction measurement, wireless charging, humidifiers, medical ultrasonography, sonar, burglar alarms, and non-destructive testing. Transmitter and receiver are two main parts of the sensor where former converts an electrical signal to ultrasonic waves while later converts that ultrasonic signals back to electrical signals. These ultrasonic waves are nothing but sound signals that can be measured and displayed at the receiving end. It gives precise measurement details and comes with accuracy (resolution) around 3mm, terming there might be a slight difference in the calculated distance from the object and the actual distance.
Following formula is used to calculate the distance of the object. S = (V x t)/2
Where S is the required distance, V is the speed of sound and t is the time sound waves take to come back after hitting the object. We need to divide the value by 2 because time will be double as the waves travel and bounce back from the initial point. Dividing it by 2 will give the actual distance of the target object.
SOFTWARE USED: • ARDUINO IDE • PYTHON IDLE(Python 2) • Python Serial Library • Python PyAutoGUI Library • Python opencv library ARDUINO IDE is integrated development environment (IDE) is a cross-platform application that is written in the programming language Java. The Arduino IDE supplies a software library from the Wiring project, which provides many common input and output procedures. IDLE is Integrated Development and Learning Environment. PySerial is a Python API module which is used to read and write serial data to Arduino or any other Microcontroller. The purpose of PyAutoGUI is to provide a cross-platform Python module for GUI automation for human beings. The API is designed to be as simple as possible with sensible defaults.