I was able to run the Linear flow example. One thing that bothered me was that there are no well-defined "input" and "output" data for the tasks I can visualize in the UI. For example.

@kubernetes(secrets="s3", memory=500, cpu=0.1)
    @step
    def start(self):
        self.my_var = "hello world"
        self.next(self.a)

    @kubernetes(secrets="s3", memory=500, cpu=0.1)
    @step
    def a(self):
        print("the data artifact is: %s" % self.my_var)
        self.next(self.end)

In the above snippet,

start

created

my_var

and it was used in

a

. I cannot find the visualization of this dataflow in the UI. Is this intended? I have added a reference screenshot from Flyte as an example.

By default the UI is more intended to visualize execution details, rather than granting full access to the underlying data. There are a few features built-in that help with visualizing the data though: • using

cards

as part of the flow, these are accessible through the UI as well ◦ https://docs.metaflow.org/metaflow/visualizing-results • enabling

artifact search

for the UI. This will allow filtering the tasks on the timeline by artifact name&value ◦ enable this by adding the

FEATURE_ARTIFACT_SEARCH=1

environment variable for the ui backend service deployment. • you can also enable an

artifact table

for the task view, which will try to list artifacts and their values with a best-effort basis (some values don't translate well to text) ◦ enable this by adding the

FEATURE_ARTIFACT_TABLE=1

environment variable for the UI backend service

enable this by adding the FEATURE_ARTIFACT_TABLE=1 environment variable for the UI backend service

Let me try this.

Excellent. you can see the search box under the tabs as well. It should have an autocomplete for artifact names, but not values. Syntax for searching if I recall correctly was

my_var:hello world