1

I have 3 tables as:

First Table: (Purchase)

date (mm-dd)    quantity   p_id
    05-05          3         1
    05-06          2         1

Second Table: (Sales)

date (mm-dd)    quantity   p_id
    05-07          1         1

Third Table: (Expired)

date (mm-dd)    quantity   p_id
    05-08          4         1

Now what I want is get details of products that have expired as:

  • When the product that has expired was purchased (FIFO)
  • Product that was purchased first, will expire/sell first.

The Output shall be:

purchase_date      expired_date      quantity    p_id
  05-05               05-08             2         1
  05-06               05-08             2         1

Explanation,

  • Store have total of 5 products by 05-06 with p_id: 1,
  • then on 05-07 1 quantity was sold of p_id: 1 i.e. product that was received on 05-05 was sold first as per FIFO

so now product we have are: (only for visualization)

date (mm-dd)    quantity   p_id
    05-05          2         1   -its 1 quantity is sold
    05-06          2         1

Then Expiry is made on 05-08, expired products are: (by FIFO)

purchase_date      expired_date      quantity   p_id
  05-05               05-08             2         1
  05-06               05-08             2         1

i.e. 2 products of 05-05 have expired and 2 of 05-06

By now logic I am implementing is:

All Appended transactions:

date (mm-dd)    quantity   p_id    expired
    05-05          3         1      False
    05-06          2         1      False
    05-07         -1         1      False
    05-08         -4         1      True

Append all transactions as: (as shown above)
Set quantity of sell and expiry as negative, and an expired variable, True only if its expiry transaction

Algorithm: (using deque push pop concept, when expiry, log it)

  1. for each group of p_id transactions:
  2. initialize a deque (empty)
  3. for each transaction (all appended transactions)
  4. if quantity is positive, push in deque
  5. else pop,
  6. if no pop up skip (as sell can be more than purchase)
  7. else if expired is false i.e. is a sell transactions,
  8. pop until difference of sell and pop is greater than equals to zero
  9. else, it is a wastage entry
  10. pop until difference is greater than equal to zero, also log each pop up with difference

*I stop when difference is greater than or equals zero signifies more quantity was bought than sold, as sold quantity is negative, consider:

bought = 2
sold = -4
diff = 2-4 = -2
therefore we need to loop for next purchase until diff >= 0

Actually currently how I am doing all this is loading all this data from my RDBMS into pandas dataframe and then applying some stuff, obviously this isn't maintainable, I want to do it within database itself as database is optimized, I want to do a complex MySQL query that does some sub-queries to get me the desired result.

When sell is made, I have different stuff going on so I want minimum load that time.

Basically what happen is:

  • A store buy product, purchase entry is made,
  • A store sell product, sell entry is made,
  • A store tells this product has expired, an expiry entry is made

*There is no relations in between them.

Also I am using Django Querysets, to get data so if I could use that, it would be great help!

Also I am open to make changes or use a totally different tool to achieve such.

Aashish Gahlawat
  • 409
  • 1
  • 7
  • 25
  • 1
    Are you happening to use, or consider using, [MySQL-8.0+](https://mysqlserverteam.com/mysql-8-0-labs-recursive-common-table-expressions-in-mysql-ctes/) or [MariaDB-10.2+](https://mariadb.com/kb/en/library/recursive-common-table-expressions-overview/) that support Recursive CTEs? There might be a way there. In addition to these adjustment tables, you might need a stock_take table having absolute numbers or every query will need to fully scan all 3 tables, but maybe this isn't a problem. – danblack Sep 14 '18 at 06:25
  • Is a replication slave for analysis possible (8.0 slave from 5.7 master is possible)? [MSSQL recursive example](https://stackoverflow.com/questions/22936112/fifo-implementation-in-inventory-using-sql) that might hint. Interested in creating a [db-fiddle](https://www.db-fiddle.com/) with sample data in MySQL-8.0 to share? – danblack Sep 14 '18 at 07:00
  • I have innodb 5.7.21, actually I need this data for some purpose and all I can do in alter/create some table, but not allowed to change db version. – Aashish Gahlawat Sep 14 '18 at 07:18
  • Sorry sir as am not much familiar with much of database terms, just started up, couldn't get all of you but possibly the example link will help out – Aashish Gahlawat Sep 14 '18 at 07:20
  • @danblack there the data is maintained in FIFO, how can I maintain that sir? – Aashish Gahlawat Sep 14 '18 at 07:30
  • A [replication slave](https://dev.mysql.com/doc/refman/8.0/en/replication.html) would be a second mysql server instance that gets updated based on what changes occur in the 5.7.21 instance. As you are doing only read access on the the 8.0 slave to perform CTE, this can occur without changing anything on the 5.7.21 master instance. The FIFO gets maintained as a temporary table like the [result table](https://mariadb.com/kb/en/library/recursive-common-table-expressions-overview/) described in the MariaDB overview. – danblack Sep 14 '18 at 07:33
  • FWIW, I started to look at how to do this with CTEs. Looking at purchases and expired only it started to look like the data I put in [this bug report](https://jira.mariadb.org/browse/MDEV-17201). So MariaDB can only handle one expired product at a time, and MySQL-8.0 cannot handling the `ORDER BY`/`LIMIT` in a recursive CTE. – danblack Sep 15 '18 at 08:17
  • Well the mariadb bug report above was fixed and will be included in the next 10.2/10.3 release. The recursive CTE in the bug report works by counting down (in purchase id) outstanding expired quantities as it works its way backwards through the purchases. Still a bit more work involved to make it account for Sales and other Expired items. – danblack Sep 18 '18 at 23:42
  • @danblack so sir, there is no workaround to do so in `mysql` by now! Shall I stick to my old `deque` implementation? – Aashish Gahlawat Sep 19 '18 at 06:25

0 Answers0