Hi all, Wanted to play around with the experimental

@pyspark

decorator. The documentation is a bit lacking (understandably so since it's experimental), but looks like it's a Metaflow extension. Was also looking into

awswrangler

as well, which works quite well too. I tried doing

python setup.py install

, but I get the following error:

RuntimeError: Conflicts in 'metaflow_extensions' files:
    Packages 'metaflow-pyspark[pyspark]', and '/opt/conda/envs/pyspark/lib/python3.10/site-packages/metaflow_pyspark-0.0.1-py3.10.egg/metaflow_extensions[pyspark]' define the same configuration module 'metaflow_extensions.pyspark.plugins.__init__'