Writing helper functions / code / interviews in docassemble, directly to data directory (i.e. bypassing the playground)?

Question

In the answer to this question it was suggested that advanced users might like to develop packages directly, rather than developing in the playground and then creating separate packages -- specifically, the idea was to create cross-package helper functions without having to package / repackage, etc.

A few questions here. Some of this is pretty "out of the box," so I'm happy to do my own experimentation / research, but just wanted to ask in case there are any known issues / thoughts / best practices that come to mind.

1. Is this the correct directory to be looking at for that suggestion: /usr/share/docassemble/local3.8/lib/python3.8/site-packages/docassemble/[[package-name]]/data?
2. Is there any magic to that directory, or can I use .../site-packages/docassemble/helpers for development of helper functions / cross-use interview stuff, and then import into .../data as needed?
3. Is there a listing of the docassemble file structure somewhere in the docs (i.e. when you run the container)? There are various references to (eg.) /usr/share/docassemble/config/, .../docassemble/backup, etc., in the docs, but I haven't found anything for the whole filesystem, or the docassemble layer's filesystem (obviously not necessary to lay out alpine, for example).
4. Looking at the "Anatomy of a docassemble package" page of the docs, along with the "Modules folder" page, I think the CWD for the purposes of imports to the .yml files is .../data?
5. If that's right, then if I were to create a /usr/share/docassemble/local3.8/lib/python3.8/site-packages/docassemble/helpers/helper.py directory and file, I could then use something like the following to import helper.py in a fictional interview.yml file? Not saying that would be the best idea, but just asking if it might work to try to understand a bit more about docassemble... haven't fully thought through the workflow for something like this (i.e. if it might be better to just use the playground to package things up, or if there's another easier option).

---
modules:
    - .....helpers.helper
---

6. Presumably then I could also take the suggestion elsewhere in the docs and abstract away the code within the package by placing it in the docassemble/packagename/data/ folder and import like this:

---
modules:
    - .module
---

I suppose I could even even structure my files like this .../data/modules/module.py, and then import with something like the following?

---
modules:
    - .modules.module
---

Happy to do my own experimentation / research, but just wanted to ask in case there are any known issues / thoughts that come to mind.

Answer 1

1. The Dockerfile runs python3.8 -m venv --copies /usr/share/docassemble/local3.8. When it comes time to upgrade to Python 3.9, it will do python3.9 -m venv --copies /usr/share/docassemble/local3.9. How packages are installed inside that virtual environment is up to the developers of Python and pip.
2. The directory site-packages/docassemble/helpers would be the directory for the Python package docassemble.helpers. You could write such a package and publish it on PyPI (the name is not taken yet).
3. You could look at the Installation page and/or the Dockerfile.
4. When you refer to data files in Python packages you can use a fully-qualified reference such as docassemble.familylaw:data/questions/kids.yml. If you write data/static/tweaks.css it will assume you mean a file in the current package. If you just give it a file name (e.g., tweaks.css) it will assume you mean a file in the current package and it will make an assumption about what data folder you are referring to based on the context. If your Python package is called docassemble.familylaw, then the data folder is located at docassemble/familylaw/data inside the docassemble.familylaw package.
5. You could make a Python package called docassemble.helpers and install it, but bypassing docassemble's package installation system by writing files directly to the Python virtual environment is not advisable. Note that docassemble is a cloud application, so a docassemble "server" may be a cluster of machines. The package management system ensures that you can do docker stop, docker rm, docker pull, and docker run, and your server will be up and running again with all the right Python packages, even if the docker pull results in an upgrade from Python 3.8 to Python 3.9.
6. The modules block has nothing to do with the data folder. If docassemble.xyz is listed in a modules block, docassemble runs from docassemble.xyz import * and imports the module's exported names into the interview answer namespace.