https://outerbounds.com/ logo
Join Slack
Powered by
This message was deleted.
# ask-metaflow
m
This message was deleted.
l
seems like some attribute you set to 
self
is not JSON parsable as it is of type 
date
. Can you convert that to a string and set it to 
self
?
g
That's odd because I'm not having this issue in any steps before the foreach join (merge) and all I add is a df of which I can retrieve using the Run API programmatically without any issue. I've also inspected all of the task artifacts and none of them have a date from what I see.
Copy code
@conda(
        libraries={
            'conda-forge::pandas': '1.3',
        }
    )
    @batch(cpu=6, memory=60000, image=IMAGE)  # `r5.2xlarge`
    @retry(times=1)
    @card
    @step
    def merge(self, inputs):
        import pandas as pd

        self.df = pd.concat(
            [s.input for s in inputs], ignore_index=True, axis=0
        )

        print('df shape', self.df.shape)

        self.merge_artifacts(inputs, include=['config'])
        self.next(self.end)
l
thanks for more details. let me take a look
few more questions: 1. Since you are creating a Dataframe in the join step, do cards get created for which ever step precedes the join (i.e. the steps that create the dataframe to be joined)? 2. You can create cards outside the a flow by calling the 
card create
command: a. you can call : 
python myflow.py card create <runid>/<stepname>/<taskid> --type default
for creating a new card of 
default
type. b. can you create the card again on the CLI and do you get the same error message ? 3. It maybe possible that the dataframe has some 
date
related type that doesn't get parsed when we parse dataframes.
g
Thanks. Yes. Number 3 was an issue. My bad! Resolved.
Removing message - just noise.
Open in Slack
PreviousNext
Powered by