Education

Greenport student creates Augmented Reality Sandbox for East End Seaport Museum

At first glance, as sunlight shines into the open garage, the piece of wooden construction appears like nothing more than an elevated sandbox.

As the garage door closes, and the room darkens, Jack McInnis flips a switch to reveal an astounding display of colors. The pure white sand suddenly transforms into swaths of blue, green, yellow, red and orange. A small lighthouse centered in the middle of the sandbox serves as the centerpiece of the virtual topography. Simulated water surrounds the lighthouse “island” and topographic contour lines show points of elevation with dark red symbolizing the highest peaks.

The Augmented Reality Sandbox will soon be on display at the East End Seaport Museum and Marine Foundation in Greenport, a nifty new exhibit that allows for hands-on learning of geography and more.

“It really does look like actual water,” said Bob McInnis, Jack’s father, as he swept his hand through the sand, instantly changing the topography display.

The creation of the sandbox took a team effort led by Jack, who is just finishing his freshman year at Greenport High School.

“It’s really a cool little thing,” Jack said, showing how holding his hand above the sand can simulate a rain effect.

Paul Kreiling, chairman of the board at the museum, said the sandbox can serve as an educational tool in many ways, including learning about erosion, rising sea levels, dredging or navigation.

“Plus you can just play in the sandbox,” he said. “I do believe the technology is wonderful. I think it will be a lasting exhibit.”

Jack McInnis, who’s finishing his freshman year at Greenport High School, set up all the technical aspects of the sandbox. (Credit: Joe Werkmeister)

To create the simulated topography requires much more than a box of sand. That’s where Jack’s expertise came in.

Relying on a blueprint published online by University of California, Davis’ W.M. Keck Center for Active Visualization in the Earth Sciences, Jack began to assemble the pieces needed to create the display. Some of parts he made with use of a 3D printer. He repurposed a depth camera from an old Xbox. And a desktop computer tucked underneath powers the simulations that allows a projector pointing down at the sandbox to create the bright images. He plans to use the 3-D printer to make an additional lighthouse that replicate Orient Point Lighthouse and Bug Light.

The sandbox prototype was created by Peter Gold, a project specialist of the UC Davis Department of Geology. The blueprints and instructions are now publicly available, although making it all work still requires extensive effort.

A similar sandbox is on display at the Long Island Science Center in Riverhead.

Jack said the desktop computer had to be wiped clean to install the software needed to power the augmented reality display. Calibrating the entire system and working to get the projector at the correct height presented tough challenges.

Jack said he ran into trouble at one point when he couldn’t get the drivers to work.

An overhead view of the topography shown in the sandbox, which can be constantly adjusted to create new layouts. (Credit: Joe Werkmeister)

“Basically halfway through setting it up, it just wouldn’t let me download anything,” Jack said. “I couldn’t even get Wi-Fi. Nothing I did could fix it.”

He consulted online with some experts on the technology and, after a back-and-forth, they figured out how the settings of one component essentially needed a switch flipped. Jack’s father compared it to how an Apple computer may not let a user download a file that may be deemed suspicious and it needs to be manually overridden.

Construction of the sandbox itself was a pleasant surprise. A family-run cabinet company donated materials and labor to assemble the legs, box and electronic support.

“We gave them the dimensions because they were curious and they came in with this sand table,” Jack said. “It’s really nice.”

The people behind the construction and donations wished to remain anonymous.

The assembled sandbox arrived at the McInnis family’s Greenport home a few weeks ago, much to their surprise at how it was all assembled with high-quality material. The same people donated money needed to secure the remaining parts. The McInnises had originally set out with a $2,000 budget, but the costs were likely higher than that, Mr. McInnis said. The projector, for example, was about $650.

The idea for the sandbox at the museum actually dates back nearly two years, and was put on halt due to the pandemic. Mr. Kreiling said Bob Jester, a retired Riverhead High School science teacher who lives in Greenport, had pitched the project as a way to spruce up the children’s room, which is known as “Captain Bob’s Quarterdeck.” Mr. Jester also knows the McInnis family well, having worked with their oldest son Joe on a multi-year research project related to honeybees.

“The McInnis family are quite the eclectic talents,” Mr. Kreiling said.

Joe McInnis was the Greenport valedictorian in 2020 and his sister Emily followed with the same honor a year later. Emily used her expertise during the pandemic on a project to create her own version of a ventilator, which was a critical tool for patients suffering early on with COVID-19. Both Joe and Emily attend Dartmouth College.

For Jack, the next challenge with the sandbox will be transporting the finished product to the nearby museum. They’ll likely shovel the sand back into bags, disassemble some of the components and then set it up once more at its final location. The sandbox should be set up sometime this month and be ready for when the museum expands its hours starting in July.

Mr. Kreiling said an exact location in the museum hasn’t been set yet, but there are a couple of options where it can fit and be made dark enough to give it the full effect.

“Ninth grade, to do all the tech on it, I’m totally impressed,” Mr. Kreiling said of Jack.