ej.microai

Class MLInferenceEngine

java.lang.Object

ej.microai.MLInferenceEngine

All Implemented Interfaces:

Closeable, AutoCloseable

public class MLInferenceEngine
extends Object
implements Closeable

The MLInferenceEngine class provides essential functionality for interacting with Machine Learning models, specifically for running inferences.

It allows users to load a trained model from a resource (using MLInferenceEngine#MLInferenceEngine(String)) or from an InputStream (using MLInferenceEngine#MLInferenceEngine(InputStream)).

Users can set input tensors, execute inference, and retrieve output tensor values.

A MLInferenceEngine allocates native resources when it is opened. The MLInferenceEngine should be closed with the close() method in order to free the native allocation.

Constructor Summary

Constructors

Constructor and Description
MLInferenceEngine(InputStream is, int inferenceMemoryPoolSize) Initializes the model from a given InputStream.
MLInferenceEngine(String modelPath, int inferenceMemoryPoolSize) Initializes the model from a pre-trained model resource.

Method Summary

Modifier and Type	Method and Description
void	close() Closes this model and its associated resources.
InputTensor	getInputTensor(int index) Gets the input tensor specified by the index.
int	getInputTensorCount() Gets the number of input Tensors.
OutputTensor	getOutputTensor(int index) Gets the output tensor specified by the index.
int	getOutputTensorCount() Gets the number of output Tensors.
boolean	isClosed() Returns whether this model has been closed.
int	reset() Resets the state of the model interpreter.
int	run() Runs an inference on the model.

Methods inherited from class java.lang.Object

clone, equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MLInferenceEngine

public MLInferenceEngine(InputStream is,
                         int inferenceMemoryPoolSize)
                  throws IOException

Initializes the model from a given InputStream.

This method will block until the model is completely retrieved in the native side from the input stream or an IOException occurs.

Map the model into a usable data structure, build an interpreter to run the model with and allocate memory for the model's tensors.

Parameters:

is - the input stream to read model from.

inferenceMemoryPoolSize - Memory allocated to load the tensors and perform inferences.

Throws:

IOException - if an IOException is encountered by the input stream.

MLInferenceEngine

public MLInferenceEngine(String modelPath,
                         int inferenceMemoryPoolSize)

Initializes the model from a pre-trained model resource.

Map the model into a usable data structure, build an interpreter to run the model with and allocate memory for the model's tensors.

Parameters:

modelPath - the path of the pre-trained model resource.

inferenceMemoryPoolSize - Memory allocated to load the tensors and perform inferences.

Method Detail

close

public void close()

Closes this model and its associated resources.

This method releases the native resources allocated when opening this model. Calling this method on a model that has already been closed has no effect.

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

getInputTensor

public InputTensor getInputTensor(int index)

Gets the input tensor specified by the index.

Parameters:

index - the index of the tensor, usually index = 0 except models who have more than one tensor as input.

Returns:

the input tensor specified by index.

Throws:

IllegalArgumentException - if index is invalid.

getInputTensorCount

public int getInputTensorCount()

Gets the number of input Tensors.

Returns:

the number of input Tensors.

getOutputTensor

public OutputTensor getOutputTensor(int index)

Gets the output tensor specified by the index.

Parameters:

index - the index of the tensor, usually index = 0 except models who have more than one tensor as output.

Returns:

the output tensor specified by index.

Throws:

IllegalArgumentException - if index is invalid.

getOutputTensorCount

public int getOutputTensorCount()

Gets the number of output Tensors.

Returns:

the number of output Tensors.

isClosed

public boolean isClosed()

Returns whether this model has been closed.

Returns:

whether this model has been closed.

reset

public int reset()

Resets the state of the model interpreter.

Returns:

0 if the execution succeeds, 1 otherwise.

run

public int run()

Runs an inference on the model.

Returns:

0 if the execution succeeds, 1 otherwise.