https://outerbounds.com/ logo
Join Slack
Powered by
Question on metaflow configs. We have a lot of log...
# ask-metaflow
l
Question on metaflow configs. We have a lot of logic here reading and validating our 
Config
for the flow. It's a lot of calls to 
self.
and some `assert`ions. I feel like this is what Pydantic is for. Is there an easy way to 1. hydrate a dataclass or pydantic model 2. then store that on 
self.
3. and reference it in later steps 4. such that we still get autocompletion on 
self.cfg
(or whatever we call it) 5. without having to put a 
from my_utils import SpecialPydanticFlowConfig
in every step?
✅ 1
a
something like this?
🚀 1
yayfox 1
thankyou 1
l
Is there a way around having to define the pydantic model nested in-line in that function? I guess put it in another file and have the import reference that. Would you get autocomplation further down in the steps though... 🤔
a
sorry, in-line in which function?
l
message has been deleted
Oh... and wait, doesn't this kill autocompletion here:
The type for 
self.config
isn't known until runtime, so you don't get dropdown suggestions after the 
.
in 
self.config.
But if you could do something like
Copy code
class PydanticConfigFlow(FlowSpec):
    config: MyConfigModel = Config("config", parser=pydantic_parser)
Then it'd work.
Except, 
mypy
might get mad.
a
correct
l
lol, can you think of a potential workaround we could do to get autocompletion back?
Looking for something that does 1. validation 2. autocompletion
a
inside the step - this would work -
Copy code
cfg: ConfigSchema = self.config
print(cfg.id)
l
Hahaha, I guess the 3rd request: 3. that wouldn't require us to add yet another import statement to all of our steps 😛
😂 1
Could 
metaflow
somehow support a generic type? Like have 
self.config
be of type 
Config[T]
where T is the return type of 
parser
a
let me hack on something and get back to you after my meetings?
❤️ 1
🤣 1
l
thanks man
🫶🏼 1
If this is the answer, I will cry lolsob
d
does that work?
l
I don't htink so , since you can't use 
.
to access attributes on the 
self.config
d
you can’t? You should be able to access stuff in 
self.config
using both 
.
and 
[]
notation. What error deos it give you (caveat: of course, with 
.
notation, the name after the dot needs to be a valid python identifier).
r
fwiw it works for me 🤷
l
1. I was wrong 2. @refined-insurance-51213 is a stud cc: @fancy-vr-43603
5 Views
Open in Slack
PreviousNext
Powered by