Where can I see erd for this : <https://outerboun...
# ask-metaflow
s
s
by erd, do you mean a diagram like below or something else? https://outerbounds-community.slack.com/archives/C02116BBNTU/p1668178267655569
s
look amazing but when the event are save , the event came from the website base on the past we predict their behavior so what I ask is : 1.until that sagemaker run the model where the events are save ? maybe in dynaodb ? 2.what will happen to the events if the scenario is that some events coma from visitots that didnt login?
v
good questions! If there isn't a huge volume of events, you can just store them in a database like RDS (which can be much cheaper than DynamoDB, if you don't need real-time access to past data). You can then export the data for featurization and training from the database. Regarding your second question, you can design your schema so that it accounts for both signed-in and anonymous users. If you need real-time predictions, you can place your feature transformation in a Python module that is shared between real-time inference and training to guarantee consistency
s
I just to make sure : the exiat data before the streaming that we use for sample is typically in DynamoDB ( in biq volume) the new data the events is on DynamoDB (in big volume ) or on rds for small volume . I read that the data is streamed and batch processed into an Apache Hadoop datastore, through an Apache Hive layer. whats better? thanks in advance