Stack Overflow
Stack Overflow

Questions tagged [datajoint]

For general questions or debugging development errors as a result of using DataJoint, a framework for scientific workflow management. For assistance with debugging, please include: Client OS, DataJoint Client (Python|MATLAB|LabBook|Pharus), DataJoint Version, and Database Version (MySQL 5.7|MariaDB 10.5|etc).

50 questions
1
vote
1 answer

Fastest way to backup / delete from table

I am trying to backup some data from a table with > 50M rows. The table looks like this # entry_id : int unsignedauto_increment # unique entry id --- time=CURRENT_TIMESTAMP : timestamp # current timestamp room …
Horst
  • 167
  • 1
  • 6
1
vote
1 answer

Datajoint: suggestions for how to handle stages needing manual review in a neuroscience data pipeline?

Wondering if there are good examples or suggestions for how to handle steps that require manual review in a database-based scientific data pipeline (datajoint). For example, we'd like to handle the pre-processing and denoising/demixing of our…
jbussell
  • 15
  • 2
1
vote
1 answer

Is there a preferred way to have a downstream table that can take data from one of two upstream tables in datajoint?

We are trying to build flexible schema where, for example, we might have single units that come from one of two places, either a SpikeSorting pipeline that begins from raw data, or imported from a file where spike sorting has already been done. We'd…
Loren Frank
  • 43
  • 2
1
vote
1 answer

Renaming/replacing primary key to query across schemas

I currently have two separate schemas for processed calcium imaging data, in which I've used two different preprocessing methods. The big goal of what I am trying to do is use sources from 1 schema as a reference and pull out the corresponding…
A__
  • 11
  • 1
1
vote
1 answer

DataJointError: Found a part table without its master table

I am using datajoint python '0.12.8'. When trying to delete from an upstream table, I get the following error message: DataJointError: Found a part table "someschema"."table_part-table" without its master table. I can preview table as well as its…
Horst
  • 167
  • 1
  • 6
1
vote
0 answers

Datajoint LabBook - can't get past login screen

I set up the datajoint lab book (docker, https://datajoint.github.io/datajoint-labbook/user.html#installation) and it seems to run. However, I can’t seem to get past the login screen (credentials / db server are correct). When I click on the green…
Horst
  • 167
  • 1
  • 6
1
vote
1 answer

Is it possible to alter a table to include reference to an upstream table in datajoint for python?

We want to alter a table to include a non-primary key reference to a new table. The old definition is: @schema class SpikeSortingParameters(dj.Manual): definition = """ # Table for holding parameters for each spike sorting run ->…
Loren Frank
  • 43
  • 2
1
vote
1 answer

Datajoint: Lock wait timeout exceeded errors

While trying to delete from a table I sometimes get Lock wait timeout exceeded errors. This seems to happen when many .populate commands (with reserve_jobs=True) are running in parallel. Removing those jobs (aborting their execution) and restarting…
Horst
  • 167
  • 1
  • 6
0
votes
1 answer

How to host a DataJoint external store on the same server as the DataJoint MySQL database

I'm trying to set up an external store to work with my pipeline. I am self hosting my database on a small server which my lab connect to. This is currently working well and we can all insert and query the database. However, we have some large arrays…
rbedford
  • 5
  • 2
0
votes
1 answer

How can I connect my DataJoint LabBook instance to my DataJoint MySQL server?

I'm trying to host DataJoint LabBook on our lab's server so our team can interact with our database. I'm afraid I don't know much about web development (I'm a datascientist) so am not finding it easy. I would like to host an instance of LabBook on…
rbedford
  • 5
  • 2
0
votes
1 answer

External file cleanup routines in datajoint

I have a lot of blob data stored externally (S3) and recalculate results regularly. I am running datajoint python 0.13.8. I have read the documentation under https://docs.datajoint.org/python/admin/5-blob-config.html#cleanup My question is about…
Horst
  • 167
  • 1
  • 6
0
votes
2 answers

Datajoint locking the table? Populating on multiple machines

I have a table that runs some heavier computation (process length ~ 5 minutes per key). I want to reserve jobs and run it on multiple machines. I noticed that computers get locked out from the table as soon as one machine starts processing a job -…
Horst
  • 167
  • 1
  • 6
0
votes
1 answer

Cannot drop AttributeAdapter tables -- "Declaration context is not set"

I have created a dj.AttributeAdapter subclass for a custom datatype (attribute_type = filepath@external). However it does not seem to be working right -- when I attempt to drop the table for instance I get the very cryptic AssertionError:…
zagaroo
  • 31
  • 4
0
votes
1 answer

Datajoint Lost Connection Error (Server connection lost due to an interface error)

I am running datapoint in Pycharm on a Mac. When I am connecting to the Database via the Pycharm Terminal, I can execute the code. Also, using the Pycharm Python Console, I am able to connect to the database. However, here, when I want to execute…
Verena
  • 1
0
votes
1 answer

Fetching data from RDS on AWS returns empty array

I have a mysql db set up on RDS on AWS (I ported the db from my personal computer), and have set up a datajoint connection to the db on Sagemaker in a Jupyter notebook. Say I have a table mouse, which has a column mouse_id. Datajoint returns the…
eric
  • 7,142
  • 12
  • 72
  • 138
1
2
3 4