Model resources

Preamble

The first step (let's call it preamble) is to prepare our Flama app, and all the ingredients needed to be able to add custom model resources. This is exactly what you have here:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
# Standard import:
import flama
from flama import Flama

# New imports (not seen so far):
from flama.models import ModelResource, ModelResourceType
from flama.resources import resource_method

# Flama app instantiation:
app = Flama(
    title="Flama ML",
    version="0.1.0",
    description="Machine learning API using Flama 🔥",
)

# Model resources will be added here
# ( ... code ... )

# In case we want to make it a runnable script:
if __name__ == "__main__":
    flama.run(flama_app="__main__:app", server_host="0.0.0.0", server_port=8080, server_reload=True)

As can be seen, there are three ingredients imported which we haven't seen yet, namely:

• ModelResource: This is a Flama built-in class which derives from the most fundamental resource class BaseResource. These classes can be seen as the interface that any custom model class will need to adhere to. In case you're not familiar with the interface design pattern, we like the definition given by RealPython:

  At a high level, an interface acts as a blueprint for designing classes. Like classes, interfaces define methods. Unlike classes, these methods are abstract. An abstract method is one that the interface simply defines. It doesn’t implement the methods. This is done by classes, which then implement the interface and give concrete meaning to the interface’s abstract methods.

• ModelResourceType: This is a Flama built-in metaclass which derives from the most fundamental resource type metaclass ResourceType. The concept of metaclass is a bit less intuitive, and some highly recognised pythonists recommend to avoid them:

  Metaclasses are deeper magic than 99% of users should ever worry about. If you wonder whether you need them, you don’t (the people who actually need them know with certainty that they need them, and don’t need an explanation about why).

  • Tim Peters

  We agree with such a claim, which is why you won't need to worry about them, apart from having to import the ModelResourceType, and use it as an argument when building your custom model class.

• resource_method: This is a Flama built-in decorator which allows us to convert a given class method into an app endpoint, with the following main arguments:

  • path: Route path through which we'll be able to call the decorated method, e.g. /predict/
  • methods: HTTP methods associated with the endpoint, e.g. GET or POST
  • name: Name which uniquely determines the route being added to the app.

Predict

We can start with one of the endpoints given for granted when using the tools showed in previous sections, namely /predict/. The way in which we build our own model resource is very straightforward, we only need to use the following pattern:

1
2
3
4
5
6
7
8
9
10
11
12
13
# FIXME: Missing schemas

class MySKModel(ModelResource, metaclass=ModelResourceType):
    # special names:
    verbose_name = "My ScikitLearn Model"
    model_path = "sklearn_model.flm"
    name = "sk_model"

    # FIXME: Missing types
    #       This won't work, but it's almost correct
    @resource_method("/predict", methods=["POST"], name="model-predict")
    def predict(self, data):
        return self.model.predict(data)

As we already said, this example is not functional (just yet). However, it contains a good amount of the pieces required, so we can start talking about these ingredients:

• MySKModel(ModelResource, metaclass=ModelResourceType): The very first thing needed is to create our own model-resource class, which means inheriting from ModelResource. However, that's only the interface, and we also need some extra magic which lives in the metaclass ModelResourceType. Only with these two ingredients, your model-resource class will be Flama compliant.

• Special attribute names: These three attributes are mandatory for several purposes. The attribute model_path specifies where to find the binary file with the model, verbose_name defines a tag for the documentation, and name is used only internally, but required to uniquely identify the resource being added.

• @resource_method: The first argument determines the path associated to the method which the decorator accompanies to, e.g. /predict/. It also receives as parameter methods, which refers to the request methods to which this endpoint responds to (e.g. POST), and name, a parameter used internally to uniquely identify this endpoint.

It's time to know why we said the previous example won't work: it's all due to missing types and schemas.

Flama is a strongly-typed-and-schema-specific framework. Wait, what? This only means that we need to provide the schema type of the input and output arguments if we want any method to be Flama compliant, and such schemas need to be registered in the app.

What do we need to make this example a working example? Let's see what the correct code looks like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
import typesystem as ts

# Definition of input (X) and ouput (Y) schemas
X = ts.Schema(fields={"input": ts.fields.Array()})
Y = ts.Schema(fields={"output": ts.fields.Array()})

# Registration of schemas:
app.schema.register_schema("X", X)
app.schema.register_schema("Y", Y)

class MySKModel(ModelResource, metaclass=ModelResourceType):
    # special names:
    verbose_name = "My ScikitLearn Model"
    model_path = "sklearn_model.flm"
    name = "sk_model"

    # Correct typed method with some documentation
    @resource_method("/predict/", methods=["POST"], name="model-predict")
    def predict(self, data: X) -> Y:
        """
        tags:
            - My ScikitLearn Model
        summary:
            Run predict method.
        description:
            This is a more detailed description of the predict method.
        responses:
            200:
                description: ML model prediction.
        """
        return {"output": self.model.predict(data["input"])}

You will notice the following changes:

• typesystem.Schemas: We've added two schemas to define the data type of the input and the output. We have used typesystem for this purpose. If you don't remember how this python package got there, go back to Quickstart. Also, remember that Flama is also compatible with Marshmallow. For more information about how to define schemas, we refer to their official documentation websites: typesystem, and marshmallow, respectively.

• app.schema.register_schema: These lines appear right after the definition of our schemas, and they are intended to register the new data schemas in the app. This makes possible the automatic identification of body arguments in request, via the design pattern known as dependency injection. The finer details on how this happens are left aside (we recommend the curious developer to dive into the code, taking advantage of the open code). In layman's terms, thanks to the registration of these schemas, Flama will be in charge of identifying what's what when a request comes accompanied by data by analysing the schema of the input data.

• predict(self, data: X) -> Y: Type hinting is mandatory, and (as we just mentioned) plays a key role in the dependency injection responsible for identifying and type-checking incoming data.

Finally, we can put all these pieces together and get a fully functional code. Thus, you should now be able to flama-run it (this is how we read the `flama run command line) after adding the previous code snippet to the code introduced in the preamble. Or, if you prefer, you can get the full example from here.

Inspect

It shouldn't be a surprise that the other endpoint we got for free before, i.e. GET /, is completely analogous to the previous example. In the following code, we introduce a reimplementation of such method under a different route in order to be able to compare both:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
class MyModel(ModelResource, metaclass=ModelResourceType):
    name = "my_model"
    verbose_name = "My ML model"
    model_path = "path/to/your_model_file.flm"

    @resource_method("/inspect/", methods=["GET"], name="model-inspect-model")
    def inspect_model(self):
        """
        tags:
            - My ScikitLearn Model
        summary:
            Get metadata info.
        description:
            This is a more detailed description of the method.
        responses:
            200:
                description: ML model info.
        """
        return {"params": self.model.inspect()}

Custom methods

And, finally, let's add something completely custom to our model-resource. We will add a method to get information about the model itself, which is not part of the model's API. This is a good example of how to add custom methods to your model-resource. In particular, the method will return metadata information about the model, such as the name of the model, the version of the library used to train it, and the version of the model itself. The code is as follows:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
class MyModel(ModelResource, metaclass=ModelResourceType):
    name = "my_model"
    verbose_name = "My ML model"
    model_path = "path/to/your_model_file.flm"

    # custom attributes
    info = {
        "model_version": "1.0.0",
        "library_version": "1.0.2",
    }

    @resource_method("/metadata/", methods=["GET"], name="metadata-method")
    def metadata(self):
        """
        tags:
            - My ScikitLearn Model
        summary:
            Get metadata info.
        description:
            This is a more detailed description of the method.
        responses:
            200:
                description: ML model metadata.
        """

        return {"metadata": {
            "built-in": {"verbose_name": self._meta.verbose_name, "name": self._meta.name},
            "custom": {**self.info, "date": datetime.now().date(), "time": datetime.now().time()},
        }}

Explicit

1
2
3
4
class MyModel(ModelResource, metaclass=ModelResourceType):
    ...

app.models.add_model_resource(path="/my_model", resource=MyModel)

Decorator

1
2
3
@app.models.model("/model")
class MyModel(ModelResource, metaclass=ModelResourceType):
    ...

With this, we have finished the implementation of our model resource. Now, we can run the app and test it. We can do so by either running the app directly from our editor, or by using the flama run command line. In the latter case, we can use the --app-debug flag to enable the debugging mode, which will allow us to use the interactive debugger in case of an error. The following code shows how to run the app using the flama run command line:

> flama run add_model_resource:app --app-debug

PyTorch

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
@app.models.model("/pytorch_model/")
class PyTorchModel(ModelResource, metaclass=ModelResourceType):
    name = "pythorch_model"
    verbose_name = "PyTorch model"
    model_path = "pytorch_model.flm"

    info = {
        "model_version": "1.0.0",
        "library_version": "1.0.2",
    }

    @resource_method("/metadata/", methods=["GET"], name="pytorch-metadata")
    def metadata(self):
        """
        tags:
            - PyTorch model
        summary:
            Get metadata info.
        description:
            Method returning relevant metadata information.
        responses:
            200:
                description: ML model metadata.
        """

        return {"metadata": {
            "built-in": {"verbose_name": self._meta.verbose_name, "name": self._meta.name},
            "custom": {**self.info, "date": datetime.now().date(), "time": datetime.now().time()},
        }}

Scikit-Learn

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
@app.models.model("/sklearn_model/")
class SKLearnModel(ModelResource, metaclass=ModelResourceType):
    name = "pythorch_model"
    verbose_name = "ScikitLearn model"
    model_path = "sklearn_model.flm"

    info = {
        "model_version": "1.0.0",
        "library_version": "1.0.2",
    }

    @resource_method("/metadata/", methods=["GET"], name="sklearn-metadata")
    def metadata(self):
        """
        tags:
            - ScikitLearn model
        summary:
            Get metadata info.
        description:
            Method returning relevant metadata information.
        responses:
            200:
                description: ML model metadata.
        """

        return {"metadata": {
            "built-in": {"verbose_name": self._meta.verbose_name, "name": self._meta.name},
            "custom": {**self.info, "date": datetime.now().date(), "time": datetime.now().time()},
        }}

TensorFlow

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
@app.models.model("/tensorflow_model/")
class SKLearnModel(ModelResource, metaclass=ModelResourceType):
    name = "tensorflow_model"
    verbose_name = "TensorFlow model"
    model_path = "tensorflow_model.flm"

    info = {
        "model_version": "1.0.0",
        "library_version": "1.0.2",
    }

    @resource_method("/metadata/", methods=["GET"], name="tensorflow-metadata")
    def metadata(self):
        """
        tags:
            - TensorFlow model
        summary:
            Get metadata info.
        description:
            Method returning relevant metadata information.
        responses:
            200:
                description: ML model metadata.
        """

        return {"metadata": {
            "built-in": {"verbose_name": self._meta.verbose_name, "name": self._meta.name},
            "custom": {**self.info, "date": datetime.now().date(), "time": datetime.now().time()},
        }}

Once you have your custom model resource implemented, you can run the app and test it. The best way to check that everything is working as expected is by navigating to the /docs/ endpoint. This will show you the documentation of the app, as well as the documentation of the custom model resource. In the following, we show the documentation of the custom model resource for the ScikitLearn model: