ROCK AND ROLL! MACHINE VISION CAMERAS FOR VR IN LIVE CONCERTS

 AUGUST, 2020  ARTICLE

Not long ago, virtual reality was little more than the stuff of science fiction books and movies. Today, virtual reality is not only making inroads in actual high-end science and technology, but also in ways that affect the life of everyday people from interactive gaming and data-driven sports broadcasting, to video conferencing, education & training, and live music & concerts.

Many music and concert lovers may have had the experience of been shoved around in the audience or had a very limited view of their favorite artists as they struggle to get the best possible view of the stage from the audience. Some people prefer to avoid the front crowds and prefer to sit back in the lawn area or back bench seats where the rowdy crowd behavior is minimal. But choosing this option means there is a risk of missing out on the details of the artists performance or the expressions conveyed to listeners within the proximity of eye contact with the performer. Such moments add to the live feel and reality of a concert experience.

In addition, since the quasi-collapse of the music industry where revenues for physical music tumbled drastically between 2001 and 2018, the industry has turned to live music as its main source of income. With several hundreds of live concerts taking place globally each year, it is impossible for fans to be physically present at all times.

Another challenge for live concerts is that they place a natural limit on the number of people who can attend them. This again, has made it harder for music fans to see their favorite artists. To meet these challenges, virtual reality has stepped in and is now playing a key role in bringing live concert experiences from the front rows to the living rooms of fans and audiences worldwide.

Virtual reality concerts are a win-win situation for the music industry and the music fans. In addition to those who pay to actually be present at a concert, the music industry can monetize everyone who couldn't obtain tickets, or who didn’t always feel like going out to see their favorite musicians perform. Virtual reality lets the music industry combine the best of both worlds: the apparent spontaneity and singularity of live music with the reproducibility and accessibility of recorded music.

Camera technology plays a key role in enabling virtual reality in live concerts. This is because the concerts are captured live from various angles using high-end cameras. The live images are then processed in almost real time to enable remotely-located audiences to choose and view the concert from positions of their choice (e.g., viewed from the front rows, viewed from different acompanists such as percussionsts, guitarists or piansts, different views of the crowds, etc.) all while delivering an immersive experience that goes far beyond that provided by a traditional concert DVD.

From a display perspective, like in sports imaging, the horizontal pixel resolution plays an important role in the quality of virtual reality. This resolution can either be actual or interpolated from a higher or lower raw image format.

4K horizontal resolution for VR has been around for quite some time. 4K, also known as Ultra HD, has a pixel resolution of 4,096 x 2,160 pixels. When compressing video streams from 4K to HD-streamable video the images are clearer, sharpner and cleaner. Shooting at such a high resolution gives editors and image processing engineers an opportunity to zoom far into images and reframe without losing information.

Using an 8K horizontal resolution, also known as Full Ultra HD, allows the user to zoom in twice as much and still get a 4K image. However, achieving real 8K horizontal resolution for virtual reality applications is difficult even if the cameras support 8K horizontal resolutions. This is because most virtual reality installations prefer each camera to have an ultra-wide field of view to give viewers a panoramic or hemispherical view while reducing the equipment handling complexities during live concerts.

The only way to achieve such ultra-wide fields of view is by using fish-eye lenses. Combining a fish-eye lens with a camera using a rectangular sensor is only possible by having an image circle that is smaller than the sensor. Today, sensors with 8K horizontal resolution are able to achieve 5324 pixels in real horizontal resolution when paired with a fish-eye lens of 4.3 mm focal length. This helps to achieve an angle of view of 250° with 21 pixels per degree, which is a good number of pixels for high quality image processing and enhancement. Interpolation can then be used to achieve an 8K horizontal screen resolution.

Obviously, the higher the camera resolution, the closer one can get to achieving real 8K horizontal resolution. But it is important to remember that these are live action events. Higher VR resolution is only useful if a camera speed of at least 30 FPS can be maintained. This limits the choice of cameras that can be used for VR applications.

One final requirement for cameras used in virtual reality applications is reliable data transmission at low noise levels over long distances. Concert venues are typically quite large, and cameras may need to be placed at locations far away from the crowds. CXP and optical interfaces (e.g., SFP+) are reliable and well-known interfaces to handle both the dist

QUALITY INSPECTION OF PHARMACEUTICALS USING HIGH SPEED MULTISPECTRAL IMAGING

 AUGUST, 2020  ARTICLE

Pharmaceutical manufacturing is a complex process which mainly deals with the manufacturing of drugs and medicines. Being a fully automated high-speed manufacturing processes, pharmaceutical production is especially challenging. It is subject to strict regulations given by public health authorities.

Defective containers, incorrect or missing medicine, mislabelling, inefficient packaging or decoloring are risks to consumers and thus different stages of the manufacturing process need to be critically inspected. In order to produce safe medicines that minimize consumer risk and succeed in a competitive market at the same time, there is a requirement for highly effective, versatile, and sensitive quality control systems. Optical quality control using camera technology plays an important role to fulfill the challenging inspection tasks in pharmaceutical manufacturing.

Pharmaceutical products come in various forms and packages, the most common being blister packages and tablets. Those consist mainly of three parts: cavity, seal, and the tablet or drug itself. The cavity is made from synthetic material or aluminium and holds the drug.

Cavity and drug are sealed with a synthetic material, aluminium, paper, or soft foil. Though each component is closely monitored prior to packaging, shortcomings still occur during the primary packaging process. Damage to the package or content, including incorrect placement, coloring, or labelling, must be identified and eventually followed by removal of the defective product.

Production numbers are extremely high for most pharmaceutics. Optical quality control systems along with sophisticated machine learning algorithms can handle large numbers, while offering high sensitivity for defect recognition. Using high speed optical control systems, the whole sample can be inspected, which is a major advantage compared to other quality control systems like manual or mechanical inspection which can end up destroying the sample during the inspection process. There are also limitations on the size of the sample that can be handled using mechanical inspection systems.

For many years, inspection of pharmaceutical packages has been carried out with conventional RGB cameras, using only visible features to detect flaws. With the advent of multispectral cameras, one can now move beyond the visible spectrum. Multispectral cameras capture information of multiple discretely positioned spectral bands, including bands outside the visible region.

In addition to visible R-G-B imaging, the additional spectral bands in multispectral imaging can assist in distinguishing different tablets based on their chemical composition, even if they are already enclosed and sealed. Furthermore, the quantity and uniformity of the active pharmaceutical component (APC) in the tablet can be measured.

The possibility to assess the extrinsic and intrinsic properties at the same time has major advantages compared to conventional quality control inspection systems. Extrinsic properties such as package condition, labelling and dosage instructions, and color coding can be inspected using the visible spectrum. Intrinsic properties of medicinal packages such as breakage of pills, fill levels of liquids, foreign objects and quantity of pills can be captured using specific spectral bands – typically in the near infrared (NIR) region.

Multispectral imaging can also be used in applications related to mistaken identities of defects. For example, in parenteral (injectable) drugs, inspection is critical to verify that there are no particles in the parenteral solution. Multispectral imaging can more easily differentiate between bubbles and particles to minimize waste while ensuring the purity of the injectable medicine.

Advanced multispectral imaging also assists in inspecting the chemical composition of pharmaceuticals. Both, intrinsic and extrinsic information can be combined for quality assessment. This allows the producer to have a single quality control setup, which is generally more robust, simpler to operate and to maintain.

Personalized medicine is going to be an important area of pharmaceuticals in the future where medicines would be manufactured based on an individual’s underlying health conditions, reaction to specific chemicals, and effectiveness for a specific patient. Camera technology combined with artificial intelligence will continue to play an important role in the quality inspection of personalized medicines.

To support high throughput in pharmaceutical production lines, modern inspection systems will need to be equipped with high speed multispectral cameras, which include the ability to inspect multiple spectral bands at high speeds simultaneously. High performance interfaces such as 10GBASE-T (10 GigE Vision) not only have the bandwidth for high frame rates but also support multi-stream output over a single cable with independent control of each waveband for separate analysis or for fusing together on the host processor.

Another important consideration is the spatial resolution of the camera device. There are a variety of multispectral techniques used in cameras. Some use pixel-level filter arrays or multiple optical paths that sacrifice spatial details for spectral diversity.

Pharmaceutical inspection systems demand high spatial resolution per channel to ensure that small defects such as cracks or foreign particles on pill surfaces, air bubbles in liquids, dosage instructions on extrinsic packaging, etc. are clearly identifiable.

Accurate alignment and overlap of the individual spectral bands assist in precisely identifying the position and size of the defects. It also helps to simultaneously trace and correlate the defect characteristics seen through different spectral bands. A multispectral camera with full sensor resolution and a single optical axis for all spectral bands is often the most precise method to achieve such results.

Lastly, builders of future pharmaceutical inspection systems will benefit from new customization technology that allows them to precisely specify the size and location of the spectral bands needed for their particular application. In this way they can keep the number of wavebands to a minimum in order to maximize the efficiency of the system. Having more spectral bands than needed can result in challenging light source requirements and can drastically reduce the speed of the multispectral system.

Vision system builders can use the customization approach to create the right balance between the number of bands, the speed of the system and effectiveness of the inspection process.