Greetings! I'm Marc Schlossberg, and I've been working alongside Danny Pimentel and graduate student Luke Walker at the University of Oregon on an exciting project: crafting augmented reality (AR) landscapes for our streets. What I have here is a practical, step-by-step guide designed to walk you through how to bring this innovative vision to your own neighborhood. So, let's roll up our sleeves and get our hands digitally dirty!
2023 marks a significant year for 3DStreet, made possible through our talented community of contributors both volunteer and paid. Zade Agency fits in the latter category as a software development team for-hire based in Ukraine. Over the past year Zade emerged as a major boost to our development productivity and I wanted to share how their contributions have helped 3DStreet in a critical time for the organization.
A few weeks ago I attended a "developer day" at a local electronics company down the road featuring their new mixed reality VR headset. I spent the entire day testing many different WebXR applications with a heavy focus on evaluating which user interaction elements will be relevant for developers.
Below are some notes intended for other 3DStreet code contributors to reference as we work on supporting new WebXR compatible devices.
3DStreet's mission is to empower anyone to visualize safer streets. A common request to support this mission is to bring in local real-life elements from your actual streets by scanning them into 3D objects. Until now, the best technology for doing this (photogrammetry to textured 3D polygons) resulted in huge file sizes, difficult to edit output files, and gooey visuals like melting trees or cars that you may see on Google Maps in 3D mode.
Enter Gaussian Splatting -- earlier this year a groundbreaking photogrammetry and visualization technique called gaussian splatting was released as part of a research paper published at SIGGRAPH 2023. I believe splatting to be the best modern method to allow everyday users like you and me to scan 3D objects with our phones and retain the fine detail of organic material especially plants, trees, flowers and other natural elements -- all things we'd like to see more of in our streets!
This is still an R&D project, not yet a supported built-in feature of 3DStreet. In this post we share research and progress on how we are improving the visual appearance of compositing new scenes from multiple splats using depth buffer and selective splat discarding in three.js and A-Frame, underlying 3D frameworks supporting 3DStreet.