Discover Metavision® Intelligence Suite

5x Award-Winning Event-Based Vision Software Toolkit.

Free to Download, Free to Build, Free to Sell

5X AWARD-WINNING EVENT-BASED VISION SOFTWARE SUITE

95 algorithms, 67 code samples and 11 ready-to-use applications in total so you can either start your discovery or build your Event-Based Vision product. Get started with the most comprehensive Event-Based Vision software toolkit to date.

OPEN SOURCE ARCHITECTURE

Metavision Intelligence Suite is based on an open source architecture, unlocking full interoperability between our software and hardware devices and enabling a fast-growing Event-Based community.  

LEARN MORE

LEADING EVENT-BASED MACHINE LEARNING TOOLKIT

Build your advanced Event-Based ML network leveraging the most performant object detector to date spotlighted at NeurIPS, the largest HD dataset, Event to Video and Video to Event pipelines, Training, Inference, grading features and more.

LEARN MORE

6 MODULE FAMILIES

With a wide range of computer vision fields covered: Machine Learning, Computer Vision, camera calibration, high-performance applications and more, the tool you are looking for is here.

EXTENSIVE DOCUMENTATION & SUPPORT

With 300+ pages of regularly updated content on docs.prophesee.ai, more than 20 Jupyter notebooks, reference data, extensive guidelines, get a head start on your product development.

GET RESULTS IN MINUTES

We took over 6 years to perfect the largest collection of pre-built pipelines, extensive datasets, code samples, GUI tools and more, so you could get results in minutes.

DOWNLOAD, BUILD, SELL - FOR FREE

 

Our software toolkit is available on Windows and Ubuntu PC. It is entirely free, without any time constraint. It includes a commercial-grade license, enabling you to design and sell your own Metavision Intelligence-based product, leveraging Prophesee Event-Based Metavision sensing technologies.

APPLICATIONS

TOOLS

HARDWARE

APPLICATIONS

DETECTION INFERENCE

Unlock the potential of Event-Based machine learning, with a set of dedicated tools providing everything you need to start execution of Deep Neural Network (DNN) with events. Leverage our pretrained automotive model written in pytorch, and experiment live detection & tracking using our c++ pipeline. Use our comprehensive python library to design your own networks.

Pretrained network trained on a 15h and 23M labels automotive dataset 

Live detection and tracking @100Hz

DETECTION TRAINING

Train your own Object Detection application with our ready-to use training framework. Experiment with multiple pre-built Event-Based Tensor representations, and training network topology suited for event-based Data.

4 pre-built tensor representation 

Automated HDF5 dataset generation 

Comprehensive training toolbox, including Tailor-made preprocessing, DataLoader,  NN architecture, visualization tool and more

VIDEO TO EVENT

Bridge frame-based and event-based worlds with our Video to Event Pipeline. Generate synthetic data to augment your dataset, and partially reuse existing references.

Off-the-shelf, ready-to-use Event simulator

GPU-compatible Event simulator seamlessly integrated 

EVENT TO VIDEO

Build grayscale images based on events with this neural network. Spare resources by facilitating automatic Event-based data annotation for tasks already solved in Frame-based. 

Off-the-shelf, ready-to-use Event to video converter

Leverage existing frame-based neural networks to obtain labels

DETECTION KPI

Evaluate your detection performance with our Object Detection KPI toolkit in line with the latest COCO API.

mAP, mAR and their variants included

OPTICAL FLOW INFERENCE

Predict optical flow from Event-Based data leveraging our pretrained Flow Model, customized data loader and collections of loss function and visualization tools to set up your flow inference pipeline. 

Self-supervised Flownet architectures 

Lightweight model

OPTICAL FLOW TRAINING

No ground truth? Leverage our self-supervised architecture. Train your Optical Flow application with our custom-built FlowNet training framework. Experiment with 4 pre-built Flow Networks tailor-made for your Event-based data.

Comprehensive flow training toolbox, including different network topologies, various loss functions and visualization mode

GESTURE CLASSIFICATION INFERENCE

Run a live Rock Paper Scissors game with our pre-trained model on live stream or on event-based recordings.

Off-the-shelf, ready-to-use classifier

GESTURE CLASSIFICATION TRAINING

Train your own object classifier with this tailor-made Event-based training pipeline.

Comprehensive training toolbox, including Tailor-made preprocessing, DataLoader,  NN architecture, visualization tool and more

VIBRATION MONITORING

Monitor vibration frequencies continuously, remotely, with pixel precision, by tracking the temporal evolution of every pixel in a scene. For each event, the pixel coordinates, the polarity of the change and the exact timestamp are recorded, thus providing a global, continuous understanding of vibration patterns.

From 1Hz to kHz range

1 Pixel Accuracy

SPATTER MONITORING

Track small particles with spatter-like motion. Thanks to the high time resolution and dynamic range of our Event-Based Vision sensor, small particles can be tracked in the most difficult and demanding environment.

Up to 200kHz tracking frequency (5µs time resolution)

>Simultaneous XYT tracking  of all particles  

HIGH-SPEED COUNTING

Count objects at unprecedented speeds, high accuracy, generating less data and without any motion blur. Objects are counted as they pass through the field of view, triggering each pixel independently as the object goes by.

>1,000 Obj/s. Throughput

>99.5% Accuracy @1,000 Obj/s.

PARTICLE SIZE MONITORING

Control, count and measure the size of objects moving at very high speed in a channel or a conveyor. Get instantaneous quality statistics in your production line, to control your process.

Up to 500,000 pixels/second speed 

99.9% Counting precision

OBJECT TRACKING

Track moving objects in the field of view. Leverage the low data-rate and sparse information provided by event-based sensors to track objects with low compute power.

Continuous tracking in time: no more “blind spots” between frame acquisitions

Native segmentation: analyze only motion, ignore the static background

CALIBRATION

Deploy your applications in real-life environments and control all the optical parameters of your event-based systems. Calibrate your cameras and adjust the focus with a suite of pre-built tools. Extend to your specific needs, and connect to standard calibration routines using our algorithmic bricks. 

Lens focus assessment 

Automatic intrinsics camera calibration 

 

EDGELET TRACKING

Track 3D edges and/or Fiducial markers for your AR/VR application. Benefit from the high temporal resolution of Events to increase accuracy and robustness of your edge tracking application.

Automated 3D object detection with geometrical prior 

3D object real-time tracking 

 

OPTICAL FLOW

Rediscover this fundamental computer vision building block, but with an event twist. Understand motion much more efficiently, through continuous pixel-by-pixel tracking and not sequential frame by frame analysis anymore.

17x less power compared to traditional image-based approaches 

Get features only on moving objects

XYT VISUALIZATION

Discover the power of time – space continuity for your application by visualizing your data with our XYT viewer.

See between the frames

Zoom in time and understand motion in the scene

 

DATA RATE VISUALIZATION

Understand the process of event generation over time, visualize data and generate plots with the power of python.

Ready-to-use Python environment

Interface with recordings & live cameras

ULTRA SLOW MOTION

Slow down time, down to the time-resolution equivalent of over 200,000+ frames per second, live, while generating orders of magnitude less data than traditional approaches. Understand the finest motion dynamics hiding in ultra fast and fleeting events.

Up to 200,000 fps (time resolution equivalent)

TOOLS

Metavision Studio is the perfect tool to start with, whether you own an EVK or not.

It features a Graphical User Interface allowing anyone to visualize and record data streamed by PROPHESEE-compatible Event-Based Vision systems.

It also enables you to read provided event datasets to deepen your understanding of Event-Based Vision.

 

Read, stream, tune events

Comprehensive Graphical User Interface

Main features
  • Data visualization from Prophesee-compatible Event-Based Vision systems
  • Control of data visualization (accumulation time, fps)
  • Data recording
  • Replay recorded data
  • Export to AVI video
  • Control, saving and loading of sensors settings

Metavision Studio is the perfect tool to start with, whether you own an EVK or not.

It features a Graphical User Interface allowing anyone to visualize and record data streamed by PROPHESEE-compatible Event-Based Vision systems.

It also enables you to read provided event datasets to deepen your understanding of Event-Based Vision.

 

Read, stream, tune events

Comprehensive Graphical User Interface

Main features
  • Data visualization from Prophesee-compatible Event-Based Vision systems
  • Control of data visualization (accumulation time, fps)
  • Data recording
  • Replay recorded data
  • Export to AVI video
  • Control, saving and loading of sensors settings

Metavision SDK is the largest set of Event-Based Vision algorithms accessible to date. High-performance algorithms are available via APIs, ready to go to production with Event-Based Vision applications.

The provided algorithms are available as C++ APIs for highly efficient applications and runtime execution or Python APIs giving access to the C++ algorithms, Machine Learning pipelines and more.

Develop high-performance Event-Based Vision solutions

25 c++ code samples, 44 python classes 51 algorithms, 26 python samples, 16 tutorials and 11 ready-to-use applications

Main features
  • Extensive documentation / code examples /  training & learning material available
  • Runs natively on Linux and Windows
  • Compatible with Prophesee vision systems and « Powered by Prophesee » partners products
  • A complete API that allows you to explore the full potential of Event-Based Vision in just a few lines of code
/**********************************************************************************************************************
 * Copyright (c) Prophesee S.A.                                                                                       *
 *                                                                                                                    *
 * Licensed under the Apache License, Version 2.0 (the "License");                                                    *
 * you may not use this file except in compliance with the License.                                                   *
 * You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0                                 *
 * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed   *
 * on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.                      *
 * See the License for the specific language governing permissions and limitations under the License.                 *
 **********************************************************************************************************************/
​
// This code sample demonstrate how to use the Metavision C++ SDK. The goal of this sample is to create a simple event
// counter and displayer by introducing some basic concepts of the Metavision SDK.
​
#include <metavision/sdk/driver/camera.h>
#include <metavision/sdk/base/events/event_cd.h>
#include <metavision/sdk/core/algorithms/periodic_frame_generation_algorithm.h>
#include <metavision/sdk/ui/utils/window.h>
#include <metavision/sdk/ui/utils/event_loop.h>
​
// this class will be used to analyze the events
class EventAnalyzer {
public:
    // class variables to store global information
    int global_counter                 = 0; // this will track how many events we processed
    Metavision::timestamp global_max_t = 0; // this will track the highest timestamp we processed
​
    // this function will be associated to the camera callback
    // it is used to compute statistics on the received events
    void analyze_events(const Metavision::EventCD *begin, const Metavision::EventCD *end) {
        std::cout << "----- New callback! -----" << std::endl;
​
        // time analysis
        // Note: events are ordered by timestamp in the callback, so the first event will have the lowest timestamp and
        // the last event will have the highest timestamp
        Metavision::timestamp min_t = begin->t;     // get the timestamp of the first event of this callback
        Metavision::timestamp max_t = (end - 1)->t; // get the timestamp of the last event of this callback
        global_max_t = max_t; // events are ordered by timestamp, so the current last event has the highest timestamp
​
        // counting analysis
        int counter = 0;
        for (const Metavision::EventCD *ev = begin; ev != end; ++ev) {
            ++counter; // increasing local counter
        }
        global_counter += counter; // increase global counter
​
        // report
        std::cout << "There were " << counter << " events in this callback" << std::endl;
        std::cout << "There were " << global_counter << " total events up to now." << std::endl;
        std::cout << "The current callback included events from " << min_t << " up to " << max_t << " microseconds."
                  << std::endl;
​
        std::cout << "----- End of the callback! -----" << std::endl;
    }
};
​
// main loop
int main(int argc, char *argv[]) {
    Metavision::Camera cam;       // create the camera
    EventAnalyzer event_analyzer; // create the event analyzer
​
    if (argc >= 2) {
        // if we passed a file path, open it
        cam = Metavision::Camera::from_file(argv[1]);
    } else {
        // open the first available camera
        cam = Metavision::Camera::from_first_available();
    }
​
    // to analyze the events, we add a callback that will be called periodically to give access to the latest events
    cam.cd().add_callback([&event_analyzer](const Metavision::EventCD *ev_begin, const Metavision::EventCD *ev_end) {
        event_analyzer.analyze_events(ev_begin, ev_end);
    });
​
    // to visualize the events, we will need to build frames and render them.
    // building frame will be done with a frame generator that will accumulate the events over time.
    // we need to provide it the camera resolution that we can retrieve from the camera instance
    int camera_width  = cam.geometry().width();
    int camera_height = cam.geometry().height();
​
    // we also need to choose an accumulation time and a frame rate (here of 20ms and 50 fps)
    const std::uint32_t acc = 20000;
    double fps              = 50;
​
    // now we can create our frame generator using previous variables
    auto frame_gen = Metavision::PeriodicFrameGenerationAlgorithm(camera_width, camera_height, acc, fps);
​
    // we add the callback that will pass the events to the frame generator
    cam.cd().add_callback([&](const Metavision::EventCD *begin, const Metavision::EventCD *end) {
        frame_gen.process_events(begin, end);
    });
​
    // to render the frames, we create a window using the Window class of the UI module
    Metavision::Window window("Metavision SDK Get Started", camera_width, camera_height,
                              Metavision::BaseWindow::RenderMode::BGR);
​
    // we set a callback on the windows to close it when the Escape or Q key is pressed
    window.set_keyboard_callback(
        [&window](Metavision::UIKeyEvent key, int scancode, Metavision::UIAction action, int mods) {
            if (action == Metavision::UIAction::RELEASE &&
                (key == Metavision::UIKeyEvent::KEY_ESCAPE || key == Metavision::UIKeyEvent::KEY_Q)) {
                window.set_close_flag();
            }
        });
​
    // we set a callback on the frame generator so that it calls the window object to display the generated frames
    frame_gen.set_output_callback([&](Metavision::timestamp, cv::Mat &frame) { window.show(frame); });
​
    // start the camera
    cam.start();
​
    // keep running until the camera is off, the recording is finished or the escape key was pressed
    while (cam.is_running() && !window.should_close()) {
        // we poll events (keyboard, mouse etc.) from the system with a 20ms sleep to avoid using 100% of a CPU's core
        // and we push them into the window where the callback on the escape key will ask the windows to close
        static constexpr std::int64_t kSleepPeriodMs = 20;
        Metavision::EventLoop::poll_and_dispatch(kSleepPeriodMs);
    }
​
    // the recording is finished or the user wants to quit, stop the camera.
    cam.stop();
​
    // print the global statistics
    const double length_in_seconds = event_analyzer.global_max_t / 1000000.0;
    std::cout << "There were " << event_analyzer.global_counter << " events in total." << std::endl;
    std::cout << "The total duration was " << length_in_seconds << " seconds." << std::endl;
    if (length_in_seconds >= 1) { // no need to print this statistics if the video was too short
        std::cout << "There were " << event_analyzer.global_counter / length_in_seconds
                  << " events per second on average." << std::endl;
    }
}

 

Metavision SDK is the largest set of Event-Based Vision algorithms accessible to date. High-performance algorithms are available via APIs, ready to go to production with Event-Based Vision applications.

The provided algorithms are available as C++ APIs for highly efficient applications and runtime execution or Python APIs giving access to the C++ algorithms, Machine Learning pipelines and more.

 

 

Develop high-performance Event-Based Vision solutions

32 c++ code samples, 112 python classes 63 algorithms, 47 python samples, 24 tutorials and 17 ready-to-use apps & tools

Main features
  • Extensive documentation / code examples /  training & learning material available
  • Runs natively on Linux and Windows
  • Compatible with Prophesee vision systems and « Powered by Prophesee » partners products
  • A complete API that allows you to explore the full potential of Event-Based Vision in just a few lines of code
/**********************************************************************************************************************
 * Copyright (c) Prophesee S.A.                                                                                       *
 *                                                                                                                    *
 * Licensed under the Apache License, Version 2.0 (the "License");                                                    *
 * you may not use this file except in compliance with the License.                                                   *
 * You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0                                 *
 * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed   *
 * on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.                      *
 * See the License for the specific language governing permissions and limitations under the License.                 *
 **********************************************************************************************************************/
​
// This code sample demonstrate how to use the Metavision C++ SDK. The goal of this sample is to create a simple event
// counter and displayer by introducing some basic concepts of the Metavision SDK.
​
#include <metavision/sdk/driver/camera.h>
#include <metavision/sdk/base/events/event_cd.h>
#include <metavision/sdk/core/algorithms/periodic_frame_generation_algorithm.h>
#include <metavision/sdk/ui/utils/window.h>
#include <metavision/sdk/ui/utils/event_loop.h>
​
// this class will be used to analyze the events
class EventAnalyzer {
public:
    // class variables to store global information
    int global_counter                 = 0; // this will track how many events we processed
    Metavision::timestamp global_max_t = 0; // this will track the highest timestamp we processed
​
    // this function will be associated to the camera callback
    // it is used to compute statistics on the received events
    void analyze_events(const Metavision::EventCD *begin, const Metavision::EventCD *end) {
        std::cout << "----- New callback! -----" << std::endl;
​
        // time analysis
        // Note: events are ordered by timestamp in the callback, so the first event will have the lowest timestamp and
        // the last event will have the highest timestamp
        Metavision::timestamp min_t = begin->t;     // get the timestamp of the first event of this callback
        Metavision::timestamp max_t = (end - 1)->t; // get the timestamp of the last event of this callback
        global_max_t = max_t; // events are ordered by timestamp, so the current last event has the highest timestamp
​
        // counting analysis
        int counter = 0;
        for (const Metavision::EventCD *ev = begin; ev != end; ++ev) {
            ++counter; // increasing local counter
        }
        global_counter += counter; // increase global counter
​
        // report
        std::cout << "There were " << counter << " events in this callback" << std::endl;
        std::cout << "There were " << global_counter << " total events up to now." << std::endl;
        std::cout << "The current callback included events from " << min_t << " up to " << max_t << " microseconds."
                  << std::endl;
​
        std::cout << "----- End of the callback! -----" << std::endl;
    }
};
​
// main loop
int main(int argc, char *argv[]) {
    Metavision::Camera cam;       // create the camera
    EventAnalyzer event_analyzer; // create the event analyzer
​
    if (argc >= 2) {
        // if we passed a file path, open it
        cam = Metavision::Camera::from_file(argv[1]);
    } else {
        // open the first available camera
        cam = Metavision::Camera::from_first_available();
    }
​
    // to analyze the events, we add a callback that will be called periodically to give access to the latest events
    cam.cd().add_callback([&event_analyzer](const Metavision::EventCD *ev_begin, const Metavision::EventCD *ev_end) {
        event_analyzer.analyze_events(ev_begin, ev_end);
    });
​
    // to visualize the events, we will need to build frames and render them.
    // building frame will be done with a frame generator that will accumulate the events over time.
    // we need to provide it the camera resolution that we can retrieve from the camera instance
    int camera_width  = cam.geometry().width();
    int camera_height = cam.geometry().height();
​
    // we also need to choose an accumulation time and a frame rate (here of 20ms and 50 fps)
    const std::uint32_t acc = 20000;
    double fps              = 50;
​
    // now we can create our frame generator using previous variables
    auto frame_gen = Metavision::PeriodicFrameGenerationAlgorithm(camera_width, camera_height, acc, fps);
​
    // we add the callback that will pass the events to the frame generator
    cam.cd().add_callback([&](const Metavision::EventCD *begin, const Metavision::EventCD *end) {
        frame_gen.process_events(begin, end);
    });
​
    // to render the frames, we create a window using the Window class of the UI module
    Metavision::Window window("Metavision SDK Get Started", camera_width, camera_height,
                              Metavision::BaseWindow::RenderMode::BGR);
​
    // we set a callback on the windows to close it when the Escape or Q key is pressed
    window.set_keyboard_callback(
        [&window](Metavision::UIKeyEvent key, int scancode, Metavision::UIAction action, int mods) {
            if (action == Metavision::UIAction::RELEASE &&
                (key == Metavision::UIKeyEvent::KEY_ESCAPE || key == Metavision::UIKeyEvent::KEY_Q)) {
                window.set_close_flag();
            }
        });
​
    // we set a callback on the frame generator so that it calls the window object to display the generated frames
    frame_gen.set_output_callback([&](Metavision::timestamp, cv::Mat &frame) { window.show(frame); });
​
    // start the camera
    cam.start();
​
    // keep running until the camera is off, the recording is finished or the escape key was pressed
    while (cam.is_running() && !window.should_close()) {
        // we poll events (keyboard, mouse etc.) from the system with a 20ms sleep to avoid using 100% of a CPU's core
        // and we push them into the window where the callback on the escape key will ask the windows to close
        static constexpr std::int64_t kSleepPeriodMs = 20;
        Metavision::EventLoop::poll_and_dispatch(kSleepPeriodMs);
    }
​
    // the recording is finished or the user wants to quit, stop the camera.
    cam.stop();
​
    // print the global statistics
    const double length_in_seconds = event_analyzer.global_max_t / 1000000.0;
    std::cout << "There were " << event_analyzer.global_counter << " events in total." << std::endl;
    std::cout << "The total duration was " << length_in_seconds << " seconds." << std::endl;
    if (length_in_seconds >= 1) { // no need to print this statistics if the video was too short
        std::cout << "There were " << event_analyzer.global_counter / length_in_seconds
                  << " events per second on average." << std::endl;
    }
}

 

HARDWARE

BUY COMPATIBLE HARDWARE

Prophesee Evaluation Kits, Packaged sensor and Sony IMX636, realized in collaboration with Prophesee are fully compatible with Metavision Intelligence Suite as well as Century Arks’ SilkyEvCam – Powered by Prophesee.

BUILD COMPATIBLE HARDWARE

Through an extensive partnership program, Prophesee enables vision equipment manufacturers to build their own Event-Based Vision products.

Contact us to get access to source code, develop your own hardware or integrate your vision system with Metavision Intelligence.

FAQ

Do I need to buy an EVK to start ?

You don’t necessarily need an Evaluation Kit or Event-Based Vision equipment to start your discovery. You could start with Metavision Studio and interact with provided recordings first.

Which OS are supported ?

Metavision Intelligence suite supports Linux Ubuntu 18.04 / 20.04 and Windows 10 64-bit for PC X86-64 architecture). For other OS compatibilities, contact us or review our Open Source project, OpenEB.

What can I do with the provided license ?

The suite is provided under commercial license, enabling you download, use, build and even sell your own commercial application at no cost. Read the license agreement here.

Which Event-Based Vision hardware is supported ?

The software suite is compatible with Prophesee Metavision sensors and Evaluation Kits. It can also operate with compatible third party products.

 

Do you provide Python examples ?

Yes, we provide python sample code: https://docs.prophesee.ai/stable/samples.html

DOWNLOAD FREE TODAY

DOWNLOAD, BUILD, SELL - FOR FREE

 

Our software toolkit is available on Windows and Ubuntu PC. It is entirely free, without any time constraint. It includes a commercial-grade license, enabling you to design and sell your own Metavision Intelligence-based product, leveraging Prophesee Event-Based Metavision sensing technologies.