MiruSuite is constantly evolving – not only technologically, but also driven by personal passion. One area that is particularly close to our hearts is music, especially orchestral productions. As an active amateur musician, Matti regularly performs on stage himself, and this led to a central question:
Is it possible to produce high-level concert recordings with a minimal crew – ideally with just two people? Especially for amateur and professional orchestras that cannot afford large TV crews.
In a research project, we tackled exactly this challenge. In this article, we share our experiences.
State Youth Orchestra of Saxony - The test ensemble for our research
Concert recordings are no longer just a “nice-to-have.” Orchestras – whether professional or amateur – must not only captivate their audience in the concert hall, but also reach them digitally. Budgets in the arts are often limited. Large film crews with multiple camera operators are frequently not feasible – even for established ensembles.
This is where PTZ cameras come into play. They make it possible to operate many cameras with little personnel. However, it quickly becomes clear: the technology alone does not solve the problem.
In theory, it sounds simple: program presets, automate camera movements, done. In practice, however, we encountered a number of problems:
- Too many presets: High-quality orchestral recordings often require well over 100 presets – hardly practical with the built-in tools of PTZ cameras.
- Complex operation: Adjusting presets during a live production is not feasible within the few seconds available.
- High mental load: Controlling many cameras and presets requires a lot of practice and concentration.
- Musical understanding required: Trained musicians are needed in the control room to follow the music.
- Unrealistic planning: Fully planned cut lists deliver the best results, but are very time-consuming. If planned perspectives are not possible due to blocked cameras (e.g., a musician shifts position and sits behind someone else), this approach quickly reaches its limits.
- Error-prone in live operation: If you don’t plan everything but instead decide “on the fly,” unplanned camera movements often occur when the camera operator and video switcher press buttons at the same time.
These problems clearly show that we need to adapt the workflow to successfully carry out such complex productions with PTZ cameras.
Our vision is clear: the software should not only execute, but also think along. The goal is to reduce complexity and minimize sources of error without restricting creative control. This is exactly where MiruSuite comes in.
The video control room for our test concert production: 2 people (video switcher + camera operator), 5 cameras (including 4 PTZ), a video switcher, one MiruSuite instance (including iPad for control), as well as Companion.
The foundation for music productions is fully automatic music tracking – dynamic and completely independent of timecodes. Especially in classical music, this is the only viable approach. The system adapts live to the performed tempo and provides information about upcoming cues.
Before the concert, the relevant instruments are annotated on the reference recording so they can be accessed during the performance.
How it works:
- A reference recording of the piece is stored
- The live audio signal is analyzed
- MiruSuite synchronizes both signals in real time
- Marked cues are triggered at exactly the right moment
During preparation, the relevant sections are marked once. In our case, this took about 1.25 times the concert duration. During the concert, these cues can be followed visually or via headphones. In the following demo, you can see and hear the tracking view during the concert. The tracking remained stable and reliable even with tempo fluctuations throughout the entire performance.
To reduce the complexity of presets, we developed new control concepts.
- Presets can be created, sorted, and adjusted with a single click
- Structured in a table by instrument and shot size
- Fast orientation even when switching quickly between cameras thanks to consistent arrangement
Camera view for creating, managing, and triggering presets via touch on a tablet or touchscreen
For maximum speed:
- All presets are grouped by instrument
- The operator only selects:
- Instrument
- Perspective (close-up, wide shot, etc.)
- and if necessary explicitly: 1st flute, 2nd clarinet, etc.
MiruSuite automatically selects the appropriate preset.
Important: Live cameras are never moved unintentionally. This is made possible by MiruSuite’s existing integration with various video switchers. The number and exact selection of cameras can be defined quickly. By default, the current preview camera is also not moved, so the video switcher can immediately take that perspective.
With a wink, we call it "Simple Mode": When clicking on instruments or sections, MiruSuite automatically moves to suitable presets.
Preset-based productions often appear static. With the Auto Move Director, we deliberately add movement to the image – without additional effort, simply at the push of a button by the video switcher. Two variants proved effective in testing:
- Random Move: Random, controlled movement for natural dynamics
- Preset Move: Smooth transitions between two presets of the same instrument
The result:
Significantly more dynamic visuals without additional workload for the team.
With the right tools, high-quality concert recordings with a minimal crew are realistic. At the same time, preset-based systems reach their limits in certain situations:
- changing lineups can occasionally lead to unsuitable presets with empty chairs
- musicians shifting their chairs can also result in incorrect perspectives
Here we see the next step: AI-based image recognition that automatically reacts to changes and corrects presets. MiruSuite already provides the necessary foundation for this. Nevertheless, our current workflow already enables productions that were previously not feasible.
Currently, music tracking and camera control are still separate. But what happens if we fully combine both?
- Fully programmed rundowns synchronized with classical music
- Automatic triggering and correction of presets
- Minimal manual intervention
In the end, only the creative part remains: the live cut.
In line with our vision: people make creative decisions – the software handles the rest.
We are at the beginning of a new workflow for concert productions. That’s why we are looking for partner companies who want to further develop this approach with us and test it in real productions. If you are interested, feel free to get in touch with us.
