Ever tried waving your mobile phone around in the air for a better signal? Of course you have. But if you see assistant professor Romit Roy Choudhury of Duke University in North Carolina apparently doing so, he’s not worrying about poor reception. He’s using his phone as an electronic pen to write reminder notes.
In 2005, Choudhury was a PhD student at the University of Illinois in computer science. Often forgetting things, he wanted to have a quick and easy way of jotting information down.
“I envisioned having a pen with a wireless interface and an accelerometer. My idea was to be able to write in the air with the pen and click a button to email the handwriting to my email address. Such pens were unavailable and I shelved the idea,” Choudhury said.
Since moving to Duke, he’s revisited the concept thanks to accelerometers in the latest smartphones. An accelerometer senses positional changes: it’s the device that prompts the screen image to flip from portrait to landscape when the phone is turned sideways. It also enables games with repetitive movements.
“At that point, I had the idea of using the phone as a pen, since the phone has both the accelerometer and the wireless capability,” he said. “My students jumped on to the idea and did a wonderful job of turning it into a good prototype. This first round of prototyping took around six months.”
Sensing the phone’s orientation or playing a game is trivial compared to recognizing air writing. His students started by using the phone to gesture a simple square, soon finding it was tough to achieve the right shape.
“Then, one day, after innumerable revisions to the algorithm, we suddenly saw a near-perfect square,” he said.
Working through the alphabet (and numbers) has proved challenging for Sandip Agrawal, electrical and computer engineering senior at Duke’s Pratt School of Engineering and post graduate student Ionut Constandache. The standard Nokia N95 being used for the experiments has a three-axis accelerometer to detect the X, Y and Z directions.
The students wrote a Python script to obtain 30 to 35 instantaneous acceleration readings a second and taking an average to overcome “noise.” Using a wireless link, the readings from the phone were processed on a separate computer with MATLAB, a high-level computing language for mathematical computation.
The prototype software now recognizes capital letters, numbers and simple shapes. It works by interpreting the geometric shapes and the order in which the user gestures them. An A, for example, consists of three distinct strokes. Every stroke, angle, and lift of your phone is significant if you’re trying to interpret air writing on an imaginary plane. As the phone doesn’t have a gyroscope, there are difficulties in distinguishing rotational motion from linear acceleration if the phone is turned.
At the moment, the phone wirelessly sends the recorded data to an adjacent computer for processing. It quickly returns the results to the phone as text, image and an email. You also have to write large, distinct letters — not joined-up — for the recognition to work.
“The computations are simple enough to be done on the phone. It’s just a matter of time,” Choudhury said. “We simply didn’t have time to port the code since we are working on so many interesting projects.”
So what are the remaining challenges for the device, now dubbed the PhonePoint Pen by the researchers? Correcting mistakes when writing is an obvious one: the idea is to allow users to choose a movement that means “delete,” such as several horizontal shakes. When writing on the move, the phone’s accelerometer picks up your movement as well as its own, distorting the output. Better algorithms, more sophisticated built-in accelerometers and a gyroscope would all help.
Air writing is useful for noting where you left the car at the airport or quickly jotting down an appointment time. Choudhury also thinks it might help people with poor finger control or speech problems.
So when will the researchers release the prototype software for people to try out?
“We believe that will be possible in few months,” he said. “We have the current prototype working on Nokia N95, but the technology can be used for any programmable smartphones with small tweaks.”
ROLLER-COASTER RIDE: More than five earthquakes ranging from magnitude 4.4 to 5.5 on the Richter scale shook eastern Taiwan in rapid succession yesterday afternoon Back-to-back weather fronts are forecast to hit Taiwan this week, resulting in rain across the nation in the coming days, the Central Weather Administration said yesterday, as it also warned residents in mountainous regions to be wary of landslides and rockfalls. As the first front approached, sporadic rainfall began in central and northern parts of Taiwan yesterday, the agency said, adding that rain is forecast to intensify in those regions today, while brief showers would also affect other parts of the nation. A second weather system is forecast to arrive on Thursday, bringing additional rain to the whole nation until Sunday, it
CONDITIONAL: The PRC imposes secret requirements that the funding it provides cannot be spent in states with diplomatic relations with Taiwan, Emma Reilly said China has been bribing UN officials to obtain “special benefits” and to block funding from countries that have diplomatic ties with Taiwan, a former UN employee told the British House of Commons on Tuesday. At a House of Commons Foreign Affairs Committee hearing into “international relations within the multilateral system,” former Office of the UN High Commissioner for Human Rights (OHCHR) employee Emma Reilly said in a written statement that “Beijing paid bribes to the two successive Presidents of the [UN] General Assembly” during the two-year negotiation of the Sustainable Development Goals. Another way China exercises influence within the UN Secretariat is
CHINA REACTS: The patrol and reconnaissance plane ‘transited the Taiwan Strait in international airspace,’ the 7th Fleet said, while Taipei said it saw nothing unusual The US 7th Fleet yesterday said that a US Navy P-8A Poseidon flew through the Taiwan Strait, a day after US and Chinese defense heads held their first talks since November 2022 in an effort to reduce regional tensions. The patrol and reconnaissance plane “transited the Taiwan Strait in international airspace,” the 7th Fleet said in a news release. “By operating within the Taiwan Strait in accordance with international law, the United States upholds the navigational rights and freedoms of all nations.” In a separate statement, the Ministry of National Defense said that it monitored nearby waters and airspace as the aircraft
Taiwan’s first drag queen to compete on the internationally acclaimed RuPaul’s Drag Race, Nymphia Wind (妮妃雅), was on Friday crowned the “Next Drag Superstar.” Dressed in a sparkling banana dress, Nymphia Wind swept onto the stage for the final, and stole the show. “Taiwan this is for you,” she said right after show host RuPaul announced her as the winner. “To those who feel like they don’t belong, just remember to live fearlessly and to live their truth,” she said on stage. One of the frontrunners for the past 15 episodes, the 28-year-old breezed through to the final after weeks of showcasing her unique