Researchers partner with Adobe to revolutionize visual SFX

Spiderman, Man of Steel, The Matrix, Game of Thrones, and even Boardwalk Empire wouldn’t be the same without green-screen technology. Most moviegoers recognize the signature solid green backdrops in pre-production still images and understand that the final product will have fantastic backgrounds, sizeable crowds, or special effects in the green-screen’s space. But a new paper by University of Illinois researchers  and co-authors from Adobe may change everything.

An article in Inverse Replicant explains that traditional green-screen technology is a straight-forward way to remove (and replace) the background in moving images. But computers require an easily identifiable color (like green) to identify the areas to remove. “Filmmakers need a tool that can figure out which elements of an image are ‘background,’ without the background having to be a prepared in advance.” Options exist, but are imperfect. That’s where Thomas Huang, a research professor in electrical and computer engineering, graduate student Ning Xu, and Brian Price and Scott Cohen from Adobe saw an opportunity.

Visual comparison results from ‘Deep Image Matting.’ ‘Ours-raw’ is the result of the researchers’ first stage, and ‘Ours-refined’ is the result of the second matting refinement stage.

As the authors explain in their paper, Deep Image Matting, “Previous algorithms have poor performance when an image has similar foreground and background colors or complicated textures.” Their solution is to use new algorithms based on artificial intelligence and machine learning. The result is improved results that can be used in natural settings and lead to lower special effects costs.

But the new methods aren’t just for video applications. This research has applications within Adobe’s programs like Photoshop to help designers remove the background of images.

Contact: Thomas S Huang, Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign, 217/244-1638,

Writer: Julia Megan Sullivan, Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign,