Chrome app

You can use Live Measurement in a chrome app by injecting a webview with the SDK scripts in your application and using postMessage api to communicate with the SDK frame. The archive with example application can be found here: https://developers.realeyesit.com/live/examples/chrome-app.zip

Project setup

You will need “webview” and “videoCapture” permissions in the application manifest.json:

{
  "name": "Live Measurement",
  "description": "Live Measurement Demo Chrome App",
  "version": "0.1",
  "manifest_version": 2,
  "app": {
    "background": {
      "scripts": ["background.js"]
    }
  },
  "icons": { "16": "img/realeyes_logo_16.png", "128": "img/realeyes_logo_128.png" },
  "permissions": [
    "videoCapture",
    "webview"
  ]
}

On the application page add the webview that loads the SDK page from the CDN. The accountId parameter should be appended to the webview url.

<webview id="realeyesit-sdk" src="https://datacollection2cdn.realeyesit.com/DataCollectionSDK/live/webview/index.html?accountId=aaaaaa-bbbbbb"></webview>

Allow webcam access request that you will get from a webview:

var sdkFrame = document.getElementById('realeyesit-sdk'); // webview element
sdkFrame.addEventListener('permissionrequest', function (e) {
    if (e.permission === 'media') {
        e.request.allow();
    }
});

Client API

Download the PostMessageShim add it to your package and load it on your page:

<script src='utils/Realeyesit.PostMessageShim.js'></script>

The PostMessageShim will allow you to establish a connection between your player and a webview with the SDK. You should create a client instance in the ‘contentload’ event handler of the sdk frame. CreateClient method has an optional third parameter of timeout (in milliseconds) after which the call will be rejected with a timeout error. By default it is equal to 30000 (30 seconds).

sdkFrame.addEventListener('contentload', function () {
    var client = Realeyesit.PostMessageShim.CreateClient(window, sdkFrame.contentWindow);
});

If you need to remove the webview from DOM and cleanup client resources you should call a ReleaseClient method to dispose the client window event handlers.

Realeyesit.PostMessageShim.ReleaseClient(client);

Methods

After creating a client instance you will be able to do remote procedure calls to the webview with the SDK. Every call returns a Promise instance and resolves with the result of the call or rejects with an error that can be specific to the call or a timeout error.

client.setIdentity(options)

Allows to set custom sessionId and participantId for the session. This way you can link the session/participant to a third party system. You can also set the isReviewer flag for the session, if it is set to true this session will be excluded from the collection stats. Each parameter is optional.

Arguments:

  • options: {{ sessionId: string, participantId: string, isReviewer: bool }}

Example:

client.setIdentity({
    sessionId: 'foo',
    participantId: 'bar',
    isReviewer: true
});

client.getIdentity()

Resolves with the object that contains the sessionId and a participantId (GUID strings by default).

Resolves with: {{ sessionId: string, participantId: string, isReviewer: bool }}

Example:

client.getIdentity().then(function (identity) {
    console.log(identity);
    /*  will print:
     *  {
     *      participantId: "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee",
     *      sessionId: "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee"
     *  }
     */
}, function (err) {
    // handle the error here
});

client.addPlayer()

This method should be called for each player that you have on the page, it resolves with the id of created player. You can associate those ids with your players and make RPC calls that require a sender on behalf of your players using these ids.

Resolves with: {number}

Example:

client.addPlayer().then(function (id) {
    // resolves with the id of added player
    // use this id to do future calls.
}, function (err) {
    // handle the error here
});

client.update(id, options)

This method is used to send player state to the SDK frame. To have local player in sync with the SDK this method should be called with the interval that is not bigger than 200 milliseconds. state property of options object should be one of:

-1 - ERROR
0 - UNKNOWN_STATE_CHANGE
1 - PLAY
2 - PAUSE
3 - BUFFERING
4 - ENDED
5 - CUED
7 - PLAYER_INITIALIZED

position should be the current player time position in milliseconds. videoInfo object should contain the url string, name string, extId string (optional) and duration number in milliseconds.

Arguments:

  • id: {number} - id from addPlayer() call
  • options:
{{
    state: number,
    position: number,
    videoInfo: { url: string, name: string, extId: string, duration: number }
}}

Example:

setInterval(function () {
    client.update(id, {
        state: player.getState(),
        position: player.getPosition(),
        videoInfo: player.getVideoInfo()
    });
}, 200);

client.start(id)

Starts the data collection for the provided player id.

Arguments:

  • id: {number} - id from addPlayer() call

Example:

client.start(id).then(function () {
    // recording was successfully started, you can initiate the playback here.
}, function (err) {
    // failed to start the recording
    // handle the error here
});

client.stop(id)

Stops the data collection for the provided player id. You can call this method whenever you want to stop data collection. For example on player ‘ended’ event.

Arguments:

  • id: {number} - id from addPlayer() call
  • options: {{ releaseWebcam: boolean }} - if webcam is released a restart of the collection implies a new camera access request

Example:

client.stop(id, { releaseWebcam: true }).then(function () {
    // recording was successfully stopped.
}, function (err) {
    // failed to stop the recording
    // handle the error here
});

Events

The client instance exposes EventEmitter interface (on/off methods), you can subscribe to the following set of events:

client.on('ready', function () {
    // initialize your workflow here
});
client.on('error', function () {
    // handle SDK errors
});
client.on('play', function () {
    // SDK asks to start the playback (used for autocontrolled players)
});
client.on('pause', function () {
    // SDK asks to pause the playback (used for autocontrolled players)
});
client.on('event', function (e) {
    // e.type is one of:
    //
    // 200:  'Recording error',
    // 201:  'Recording initiated',
    // 202:  'Recording started',
    // 203:  'Recording finished',
    // 204:  'Cam permission asked',
    // 205:  'Cam access allowed',
    // 206:  'Cam access denied',
    // 207:  'Cam acces timeout',
    // 208:  'Start recording error',
    // 209:  'Stop recording error',
    // 210:  'Get token error',
    // 213:  'Cam permission dismissed',
    // 214:  'Cam not found',
    // 1000: 'Error',
    // 1001: 'Initialized',
    // 1003: 'Initializing',
    // 1004: 'Unload',
    // 2003: 'Recording stop requested'
});

Example workflow

// get reference to the webview with the SDK
const sdkFrame = document.getElementById('realeyesit-sdk');

// set poll interval for the client.update() method in milliseconds
const pollInterval = 200;

// define dictionary of player states
const playerState = {
    ERROR: -1,
    UNKNOWN_STATE_CHANGE: 0,
    PLAY: 1,
    PAUSE: 2,
    BUFFERING: 3,
    ENDED: 4,
    CUED: 5,
    PLAYER_INITIALIZED: 7
};

// get a reference to the player element
const player = document.getElementById('html5-player');

// store the state of the player in this variable
let state = 0;

// track the state of the player by handling player events
player.addEventListener("playing", () => state = playerState.PLAY);
player.addEventListener("waiting", () => state = playerState.BUFFERING);
player.addEventListener("pause", () => state = playerState.PAUSE);
player.addEventListener("ended", () => state = playerState.ENDED);
player.addEventListener("seeking", () => state = playerState.PAUSE);
player.addEventListener("loadedmetadata", () => state = playerState.CUED);
player.addEventListener("error", () => state = playerState.ERROR);

// allow the camera access in a child frame
sdkFrame.addEventListener('permissionrequest', function (e) {
    if (e.permission === 'media') {
        e.request.allow();
    }
});

// handle webview 'contentload' event
sdkFrame.addEventListener('contentload', function () {
    // create a client instance
    const client = Realeyesit.PostMessageShim.CreateClient(window, sdkFrame.contentWindow);

    // set custom identity
    client.setIdentity({ sessionId: 'foo', participantId: 'bar', isReviewer: true });

    // subscribe to SDK events
    client.on('event', console.log.bind(console, 'application event:'));
    client.on('ready', function () {
        // initialize the workflow here
        client.addPlayer().then(function (id) {
            // poll the local player and send the state to the remote adapter in the frame
            const updateIntervalId = setInterval(function () {
                client.update(id, {
                    state: state,
                    position: player.currentTime * 1000 | 0,
                    videoInfo: {
                        url: player.currentSrc,
                        name: 'foo',
                        extId: 'bar',
                        duration: player.duration
                    }
                });
            }, pollInterval);

            // stop data collection on player "ended" event
            player.addEventListener('ended', function () {
                client.stop(id).then(function () {
                    // cleanup
                    clearInterval(updateIntervalId);
                    Realeyesit.PostMessageShim.ReleaseClient(client);
                    sdkFrame.parentNode.removeChild(sdkFrame);
                }, function (err) {
                    console.error('stop() error:', err);
                });
            });

            // start data collection
            client.start(id).then(function () {
                player.play();
            }, function (err) {
                console.error('start() error:', err);
            });
        }, function (err) {
            console.error('addPlayer() error:', err);
        });
    });
    client.on('error', function (err) {
        console.error('SDK load error:', err);
        // this could happen for example if the participant does not have a webcam
        // recover the flow without Realeyes data collection.
    });
});

Known issues

When video player is embedded inside the webview of the Chrome App it consumes more CPU then in browser. This can affect the recording, there are visible frame drops in recorded video when used with a player in a webview. Please use your player only on packaged pages.