How Yahoo Immersive made the Jan. 6 XR interactive

Yahoo News’s XR reporting on the one-year anniversary of the Jan. 6 insurrection at the U.S. Capitol is the most complex immersive project we have undertaken. After analyzing hundreds of photos and documenting legal proceedings in the event’s aftermath, we set out to visualize two things: How close did rioters get to elected officials in the building at the time, and what is the current status of some of the central figures involved.

Through our 3D reconstruction, we sought to establish the locations and movements of police officers, rioters, senators and Vice President Mike Pence throughout the breach of the Capitol. We gathered and evaluated more than 700 photos and videos taken by multiple media sources, including journalists for AP, Reuters, Getty, the New Yorker and HuffPost. We included in our research photos and videos captured by rioters that were later posted by ProPublica. We also examined the testimony of senators and Capitol security footage presented during the second impeachment trial of former President Donald Trump. We referenced relevant firsthand accounts from reporters and government officials collected from public interviews and records.

We did not have access to video for every location on the first, second and third floors of the Senate chambers at all times; however, by comparing time stamps on live video and security footage as well as metadata in the files of gathered photos and social media posts, we constructed a timeline of events in the Ohio Clock corridor and the Senate chamber from 2:12 p.m. ET until 2:52 p.m. ET on Jan. 6, 2021, as well as other key areas to represent the events that unfolded on the first floor and second floors.

We were unable to show the position of all rioters at all times as they spread throughout the building, which means that some of their movements may not be represented in our model. We rendered the events of the day as completely as possible based on our visual analysis and re-created only what we could support with visual, time-stamped sourcing.

For the 3D scenes, we built a 3D environment based on a floor plan of the Capitol and compared that with photos and videos of the Senate floor and hallways, available through image libraries and C-Span. We made some changes to the scale of the walls (they are lowered) for clarity and to help our users navigate the 3D model. We illustrated all doors as closed unless photo evidence showed that they were opened during the course of the Capitol breach, either by rioters or fleeing lawmakers.

Most character models used in this rendering are from a stylized library. We used a color-coding system to identify and differentiate the figures in the display. Red represents rioters, blue is for officers, white is for senators and staffers, and green is for Vice President Mike Pence. We continued to use this color scheme whenever a character or group was referenced in the 3D text. We did customize some character models for rioters who are referenced by name.

For each of the eight scenes depicted, the composition of crowds in the Ohio Clock corridor reflects photographs taken of the space from multiple angles. The representation of senators, staffers and officers in the Senate chamber and adjacent rooms is a composite with numbers and positions extrapolated after viewing videos of the chamber and evacuation process, as well as the size of the vice president’s Secret Service detail.

The 3D text cards identifying rioters reflect reporting and Department of Justice documentation of the legal status of each rioter as of Jan. 4, 2022.

This project was created by Director of XR Content Henry Keyser, with additional work by XR creators Sonny Cirasuolo, Tim Chaffee and Brian Pincus and reporting by Yahoo News journalists Dylan Stableford and Caitlin Dickson. Yahoo Immersive produces interactive projects every month that you can find here.