We support frame processors that will receive data from the camera preview stream. This is a useful feature with a wide range of applications. For example, the frames can be sent to a face detector, a QR code detector, the Firebase Machine Learning Kit, or any other frame consumer.

cameraView.addFrameProcessor(new FrameProcessor() {
    @Override
    @WorkerThread
    public void process(Frame frame) {
        byte[] data = frame.getData();
        int rotation = frame.getRotation();
        long time = frame.getTime();
        Size size = frame.getSize();
        int format = frame.getFormat();
        // Process...
    }
}

For your convenience, the FrameProcessor method is run in a background thread so you can do your job in a synchronous fashion. Once the process method returns, internally we will re-use the Frame instance and apply new data to it. So:

  • you can do your job synchronously in the process() method
  • if you must hold the Frame instance longer, use frame = frame.freeze() to get a frozen instance that will not be affected
Frame API Type Description
camera.addFrameProcessor(FrameProcessor) - Register a FrameProcessor.
frame.getData() byte[] The current preview frame, in its original orientation.
frame.getTime() long The preview timestamp, in System.currentTimeMillis() reference.
frame.getRotation() int The rotation that should be applied to the byte array in order to see what the user sees.
frame.getSize() Size The frame size, before any rotation is applied, to access data.
frame.getFormat() int The frame ImageFormat. This will always be ImageFormat.NV21 for now.
frame.freeze() Frame Clones this frame and makes it immutable. Can be expensive because requires copying the byte array.
frame.release() - Disposes the content of this frame. Should be used on frozen frames to release memory.