0

I have basic Python and Unix questions around CPU/Network monitoring. I am running an app listening to some Websockets and saving incoming data to files (Python3.6, asyncio, aiohttp and aiofiles). About 300 KB/sec receiving.

  1. Using "top", I get 100% CPU usage. Does that mean I'm at capacity and therefore at risk of losing packets, or is it just something artificial to do with async programming? Profiling shows it's spending most of the time reading Websocket data. I don't really understand how an I/O app can consume such much CPU.
  2. Is there any tools I can use to send warnings if I each CPU or Network capacity - so I can act before risking losing data? Preferably in Python, but if not I could always run a system command from Python.
jam123
  • 129
  • 1
  • 7
  • You can try [psutil](https://pypi.python.org/pypi/psutil). – CtheSky Jan 07 '18 at 03:45
  • I don't know about concrete risks, but I think reaching CPU limit is not a good thing. it's hard to say where problem can be without code, but may be you should use [asyncio.Semaphore](https://docs.python.org/3/library/asyncio-sync.html#asyncio.Semaphore) to limit number of running simultaneously tasks? – Mikhail Gerasimov Jan 07 '18 at 14:17
  • Thanks. In your experience, how common is it to reach full CPU while running async I/O? I wondered whether this is a real concern (capacity reached, risk of losing data) or just something artificial to do with async programming. I'm not sure if limiting the number of tasks is an option for me (I am listening to live data sources and don't want to miss any data). – jam123 Jan 07 '18 at 17:10

0 Answers0