1. Visualizing artifacts: In their UI, they support previewing artifacts. For example, if you save a pandas dataframe in a step, you can preview it in the UI. I don’t know that all artifacts are previewable, but they’re making an attempt to visualize some common things which I think is terrific. Dagster and ClearML have some facilities for this as well. 2. More metadata in UI: Their UI shows a. the git URL and commit SHA of the repo it came from, b. the python dependencies used, c. the docker base image used, d. info about the compute resources, e. rudimentary hyperparam logging. f. A syntax highlighted code block showing the python code for each step. I know Metaflow isn’t trying to compete with dedicated experiment trackers, but some of this data is much more relevant to Metaflow’s function as a DAG orchestrator anyway. g. ClearML even shows a git diff of any uncommitted code changes that may have been present if a DS ran the flow from their laptop—very nice. 3. Better visual DAG view. Their visual DAG shows not just the steps, but also the artifacts produced by the steps. And even shows which downstream steps consume those artifacts. Feels similar to Dagster in this way. 4. Docker-based environments. They use dynamically built docker images for step environments!! a. You still declare python dependencies in a decorator like Metaflow. Or you can also put them into a separate file—which I loved. But you also can set a Python base image. Instead of creating a venv or conda env which are then uploaded to S3 the way metaflow does, it actually builds a whole docker image for the step and pushes it to ECR. b. I could imagine this being a

@pypi_docker

decorator which builds images instead of venvs. c. Using Docker with metaflow is a burden for us because we need to create our own pipelines to build and publish images to ECR. This means that it’s difficult for example to create a DAG where every step uses a different docker image. And when you want to add even one additional library to an existing image… you can’t without publishing a whole new image (or pip installing at runtime). This encourages our team to create “super images” which attempt to contain every dependency a DS could ever need… which naturally results in images that are heavy. I’d like to point out that ClearML also uses the ZenML approach and I found it really intuitive to use—and it still abstracts the hardest parts about docker from the DS. It would also fix the limitation Metaflow currently has where you can’t install packages that don’t have wheels published (e.g. thrift) if you’re using other packages that do (e.g) pandas. This is a major pain point for us. 5. 1st class Pyproject.toml -- So far, Dagster is the only tool I know supports “proper packaging” AKA your code base having a pyproject.toml file. ClearML has a way to add a “pip install” as an “on startup” lifecycle action for that. 6. Non-class-based API:

@step

and

@pipeline

over pure functions as opposed to nesting everything under a class. I think where Outerbounds still beats ZenML Cloud, Dagster, and ClearML (at least in our workflow) is • remote workstations • automatic right sizing of step compute, • from what I could tell—better cost and compute analytics. • Possibly some of the integrations like fast data loading from S3, pyspark, DBT, etc. but I haven’t used these features yet

the default

@card

provides (1) to a degree but it can be improved (dig deeper into artifacts, don't just truncate etc)

let me know if you have other ideas on that front

(2) and (3) make sense - shouldn't be hard to address. If you need some of these items urgently, it shouldn't be hard to include them in a custom card

(4) stay tuned! 😉

(5) - supporting

requirements.txt

and

pyproject.toml

makes sense. When it comes to "proper packaging" in general, I'd argue that Metaflow goes beyond most other tool, as we snapshot the whole execution environment at the deployment time (akin to baking a Docker layer automatically), which gives much more stable environments than `pip install`'ing on the fly, which can fail in multitude of ways (been there, seen that)

(6) it's likely that we will introduce new ways to author flows (programmatically) over time

great feedback - let me know if you have ideas/preferneces how to implement any of the above items. Also, it's always fun to consider what pain points / features no tool is addressing today, going beyond the status quo 🚀