Hello team! I was wondering if it could ring a bel...
# ask-metaflow
r
Hello team! I was wondering if it could ring a bell: we are trying to display cards in our metaflow flows but we are having in the metaflow ui pod the following error - the
/tmp
seems to be well used as I can see that some data have been written:
/tmp/metaflow_client/gs.gs:/argo-workflows-artifacts-pilot.ComponentDemoFlow/98
. Here is the error in the thread. Thank you!
1
Copy code
"GET /api/flows/ComponentDemoFlow/runs/4465/steps/start/tasks/216077/logs/err?attempt_id=0&_limit=500&_page=1&_order=-row HTTP/1.1" 200 1071 "<http://metaflow.prolaio-datasci.local/ComponentDemoFlow/4465/start/216077?direction=desc&group=false&order=startTime&section=stdout>" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36"
WARNING:CacheAsyncClient:cache_data/artifact_search:StreamWriter.drain timeout, request restart: <StreamWriter transport=<_UnixWritePipeTransport fd=13 polling bufsize=9600805>>
ERROR:root:Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/root/services/ui_backend_service/ui_server.py", line 167, in <module>
    main()
  File "/root/services/ui_backend_service/ui_server.py", line 152, in main
    loop.run_forever()
  File "/usr/local/lib/python3.11/asyncio/base_events.py", line 607, in run_forever
    self._run_once()
  File "/usr/local/lib/python3.11/asyncio/base_events.py", line 1922, in _run_once
    handle._run()
  File "/usr/local/lib/python3.11/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/opt/latest/lib/python3.11/site-packages/aiohttp/web_protocol.py", line 452, in _handle_request
    resp = await request_handler(request)
  File "/opt/latest/lib/python3.11/site-packages/aiohttp/web_app.py", line 543, in _handle
    resp = await handler(request)
  File "/opt/latest/lib/python3.11/site-packages/aiohttp/web_middlewares.py", line 114, in impl
    return await handler(request)
  File "/opt/latest/lib/python3.11/site-packages/aiohttp/web_middlewares.py", line 114, in impl
    return await handler(request)
  File "/root/services/utils/__init__.py", line 89, in wrapper
    err_trace = getattr(err, 'traceback_str', None) or get_traceback_str()
  File "/root/services/utils/__init__.py", line 84, in wrapper
    return await func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/services/ui_backend_service/api/artifact.py", line 78, in get_artifacts_by_task
    return await find_records(request,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/services/ui_backend_service/api/utils.py", line 475, in find_records
    results, pagination, benchmark_result = await async_table.find_records(
                                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/services/ui_backend_service/data/db/tables/base.py", line 239, in find_records
    result = await postprocess(result, invalidate_cache=invalidate_cache)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/services/ui_backend_service/api/utils.py", line 557, in _chained
    result = await _postprocess(result, invalidate_cache=invalidate_cache)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/services/ui_backend_service/data/refiner/refinery.py", line 78, in postprocess
    data = await self.fetch_data(
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/root/services/ui_backend_service/data/refiner/refinery.py", line 29, in fetch_data
    async for event in _res.stream():
  File "/root/services/ui_backend_service/data/cache/client/cache_async_client.py", line 106, in wait_iter
    raise CacheServerUnreachable()

services.ui_backend_service.data.cache.client.cache_client.CacheServerUnreachable
s
Cc @hallowed-glass-14538
h
r
We have fixed this first error by upgrading to all recent versions for all service, ui, static ui.