3

about one year ago, we adopted Jinja2 as our template engine for python.

Since this adoption, we are dealing with constant high CPU loading.

It's normal a Jinja template compiled and cached, with simple loops and some filters use around 30% of CPU for each content generation?

For generation, only plain objects are used, without any external calls to databases, apis or complex model methods.

Templates are structured in a max-depth of 4 levels of composite includes.

There are Guidelines on how we should follow to avoid high CPU usage?

Andre Pastore
  • 2,841
  • 4
  • 33
  • 44
  • 2
    This question sounds very vague, you're asking several different questions here. If I were you I'd focus on the first question about CPU/mem usage. My answer to that would be to profile the code first and ensure that this is where the issue lies. You may be doing DB work or ORM work inside the template and that is what is causing the high CPU/memory. Without profile that or giving us more details it's all speculation. – mjallday Oct 16 '14 at 16:23
  • 1
    I can imagine that much of your code (a database-driven web app?) is I/O-bound, and the rest is relatively simple processing. 30% of CPU usage for template rendering may be fine if _everything else_ in your application is not CPU-hungry. Don't sweat over the percentage; look at the RAM usage and response times instead. – 9000 Oct 16 '14 at 16:29
  • I edited focusing on CPU usage. @9000, we allocate a dedicated machine for run 3 Jinja jobs for generate template. It's not normal for a template engine block all other jobs. Same architecture for JSP, PHP and PERL don't consume whole CPU. That's my point. – Andre Pastore Oct 16 '14 at 16:41
  • 1
    How large are your templates and/or datasets? How much wall clock time does a generation run take, and do you feel this time is inadequately long? A CPU-bound job may slow down other jobs, or at least make them feel less responsive. To avoid that, you could increase templating process's `nice` value (under *nix), or lower the priority (under Windows), – 9000 Oct 16 '14 at 16:46
  • Lists sent to render() contains 12 elements of short-medium bytes size. nice value over this processes were already tested, but it doesn't explain or solve the CPU occupation issue. It just adjust and reduce throughput of jinja job. – Andre Pastore Oct 16 '14 at 16:53
  • I will implement this model using JSP and put comparison results here. – Andre Pastore Oct 16 '14 at 16:54
  • 1
    During a refactoring over this code, I really consider what @9000 said about some IO call. There is a bug in code that is causing problems, not related to Jinja. It is a really legacy code. Sorry for my mistake on asking it before considering a bug in our platform. Best Regards – Andre Pastore Oct 17 '14 at 02:36

0 Answers0