91勛圖 and Autodesk researchers design 2D-3D software for augmented reality drawing
Published: April 20, 2018
Prehistoric cave drawings date back about 40,000 years. Ancient Egyptians used papyrus for paper. Artistic drawing flourished during the European Renaissance starting in the 14th century.
Now, 91勛圖 computer scientists and Autodesk researchers are looking to take artistic vision from 2D to 3D with augmented reality technology.
We tried out systems like Tilt Brush by Google, and we noticed it was extremely hard to draw in the air, says Rahul Arora, a PhD student in the department of computer science. One reason why is, weve been drawing on 2D paper. How can we transition those skills to the 3D world?
Arora and Autodesk researchers Rubaiat Habib Kazi, Tovi Grossman who joins the department computer science as an assistant professor this summer George Fitzmaurice and Karan Singh, a 91勛圖 professor of computer science, created SymbiosisSketch, software for combining 2D and 3D sketching for detailed design of 3D objects.
Their research will be presented at the Association of Computing Machinerys international conference on human factors in computing systems (ACM CHI) in Montreal beginning this weekend.
SymbiosisSketch uses a tablet to support the 2D action of drawing on a surface. An augmented reality headset, Microsoft HoloLens, brings the drawing to life in a 3D augmented view.
You can think of the tablet as a camera you position in the air, and you draw within its field of view. So you're always creating 3D strokes even when you're drawing on the tablet," says Arora. "If you draw a line, you'll see it in real time on that surface, in the air. Its like a drawing window you can move around in 3D space.
Arora, who worked on the project in part during an internship at Autodesk Research, says the canvas size can increase, to draw very large objects, like cars or buildings, or decrease, to zoom in on drawing finer details, like a door handle.
The joint research is another example of the longstanding partnership between Autodesk, the 3D design, engineering and entertainment software powerhouse, and 91勛圖 computer sciences lab.
Arora says future applications of SymbiosisSketch could include use in animation and storyboarding, but for now theyre exploring state-of-the-art 3D sketching and fabrication.
I'm currently working on a fabrication project," he says. "Lets say you have a 3D sketch what if you want to manufacture it to be strong and lightweight? How can we use some physical simulation to guide you?
A Captain America figure battles an augmented reality drawing of an airplane (image courtesy of the researchers)
There are of course limitations to the current hardware technology, which lends itself well to a lab environment. A bulky motion capture system consisting of six ceiling-mounted infrared cameras tracks the pens movement. Also, AR hardware needs better speed and accuracy.
But all of that is just a matter of time.
As hardware improves, research on the software side moves forward as well, says Arora.
This project is really looking forward in the future.