What is photogrammetry

The VRTogether project is characterized by the use of state of the art technology to offer high quality content and hyper realistic material. One of these technologies is named photogrammetry, and it has been used successfully for this project to digitally recreate specific environments.

But first of all, what is photogrammetry? Photogrammetry is the technique used to obtain information about objects and the environment by using recording and measuring devices, to later digitally interpret this information. Even though this technique is commonly used in fields such as topographic mapping, architecture and geology, among others, the VRTogether artistic partners found a way to apply it to the objectives of the project. This technique is also commonly used in movies and TV shows to computer-generate content in post-production.

What did we use photogrammetry for

Among others, one of the conclusions of the previously announced on this blog first Joint Business Clinic was the preparation and elaboration of different use cases that would focus on several fields. One of these fields was the preservation and showcasing of art and heritage, offering this service to museums, theatrical companies and all culture industries and institutions.

Using this technology to recreate part of the collection of a museum was just an example of how this technique can be used with the technology developed within the VRTogether project to demonstrate that museums and art exhibition rooms have the possibility to use this type of technology, not only to eventually increase their cash flow and visitors, but to achieve, among others, the following objectives:

• To preserve the museum heritage, helping its conservation and correct visualization and enjoyment.
• To preserve the history of the city in which the museum is located.
• To bring heritage closer to a diverse audience, adapting its characteristics and needs by offering an interactive experience.
• To create a product that allows to get the most out of the facilities and cultural heritage of the museum, making it attractive and unique.
• To offer experiences to groups of people, all of them being able to experiment cultural activities at the same time, allowing a positive flow of word of mouth in and out of the museum.

Even though the final use of this technology is focused on increasing the cash flow and number of visitors to the museum, this technology also allows the generation and display of new types of content, exploring the different ways in which art and culture can be preserved and presented to the audience. The inclusion of different social groups is another factor in which VR has great potential. To resume the path of sustainable competitiveness that gets us out of the current crisis, the productive system must be able to take advantage of all the potential that culture and the arts have by imagining new scenarios, future artifacts, or reinterpreting interfaces. If the artists have something in common, designers and creatives are good at making understandable in a way close to the newest concepts. This is transcendental for society and people, embracing the innovative solutions that global challenges need. VR is committed to eliminating barriers that separate the worlds of the humanities and technologies and therefore increase the competitiveness and sustainability of our innovation system. This is in line with the bet of the European Union itself, which establishes that the culture and our values are necessary to compete on a global scale with other regions from social cohesion, and thus ensure that innovation is sustainable at the economic, social and environmental levels.

Some of the most famous museums in the world are already using some VR techniques to offer their audience a new experience. TheMo’s team created one of the most beautiful VR immersive experiences for Museum Thyssen Bornemisza. The Louvre Museum and The National Museum of Natural History, both in Paris, or The Tate Modern Museum in London are just some examples of important museum refererents that are including VR in their activities. From the VRTogether consortium we wanted to continue within this field, bringing new immersive experiences.

The chosen museum to use this technique was the Víctor Balaguer Museum & Library, located in Vilanova i la Geltrú (Barcelona, Spain). Specifically, the XIX Century Art Gallery from this museum was the one selected to be immortalized with the photogrammetry technique.

First phase of the photogrammetry process

As the Victor Balaguer Museum & Library claims, the XIX Century Art Gallery gathers the big part of the founding painting and sculpture collection in an area that recreates the original environment of the Fine Arts halls of the time with high ceilings, velvety colored walls and plenty of paintings hanging on the walls.

The audience can find different works from the collection made by the Museum’s founder, Víctor Balaguer, and also several art pieces by artists like Ramon Casas, Santiago Rusiñol, Marià Fortuny, Ramon Martí Alsina or Joaquim Vayreda, representing styles such as Romanticism, Realism and Symbolism.

How did we use photogrammetry

The photogrammetry technique was applied in different stages, since the scanning process of this technology consists of several steps.

The first phase is the acquisition of images. In late August, TheMo team travelled to the museum to do this process. The captured pictures needed to have enough overlap in order to let the photogrammetry software locate all the information of the scenario and the cameras in a digital environment. As it can be seen in the following pictures, the mix of colors in the XIX Century Art Gallery was high, so all the pictures were taken in .raw format , so information couldn’t be lost in the digital recreation of the space.

The next step consists of reviewing every single picture taken of the scenario, making sure that no space is missing, so the number of pictures that need to be checked and reviewed is large. Generally, this process is done with specialized software, such as Adobe Photoshop Lightroom. During this stage of the photogrammetry process, every picture (if needed) is retouched in terms of light, hue, saturation and every other aspect that requires an extra retouch.

Normally, this process is focused on lightening those parts of the picture that may be a little bit dark, or even the opposite process, darkening parts that are overexposed. This process has to be very detail oriented, since all the digitally recreated rooms have to look natural, and lighting is one delicate step in order to make that happen. Once this review and retouch process is done, the pictures need to be exported in .jpg format, since photogrammetry softwares don’t use the previously mentioned .raw format.

Once all the pictures have been imported to the photogrammetry software, a first alignment calculation is done. This is essential in order to obtain a scattered point cloud model. The scattered point cloud model helps the technician to evaluate if more pictures are needed, if the ones chosen are the correct ones or if everyone selected is enough to continue with the process. The following gif illustrates this paragraph that is common to every photogrammetry process.

Once this step is done, a second and more precise calculation is needed to obtain a polygon mesh model of every part of the room. This process is heavier and slower to perform, but it helps to bring naturality and hyper-realism to the final product. Since it is a step in which every piece is defined, it offers as well a solid vision of how the final model is going to look like.

Finally, and once the polygonal model is obtained, the textures of the model are calculated in order to edit the final light and color information. The following pictures show the final look of the digital recreation using this photogrammetric technique. It can be appreciated all the details of every single piece of art, obtaining the desired hyper-realistic look in order to be used for the previously mentioned use case.