1

I have a table of historical position tenures held by members within an organisation (tbl_tenue). Each position may only be held by one person at a time but one person may hold several positions sequentially or concurrently. I want to check the integrity of this table by taking each position tenure record in turn and then comparing it with every other record in the table looking for (erronous) overlaps with other tenures of the same position.

I have written a working query (below) that returns details of overlaps when passed a test_tenure_id, position_id, and the start and end dates of a particular tenure (see below) For each overlap. this query returns tenure_id, member_id, member_sn, date_started, date_ended and reason for the overlap.

Can anyone help me with the sql to now run this query against the same tbl_tenue table, each time passing it the data from one row in tbl_tenue so I can test each record for overlap with every other one (except itself of course) and return data from the record and all its overlaps?

(I realise that if I can do this then I should be able to avoid passing in the tenure_id to the WHERE clause by using the join and also avoid passing the dates as well using the join but I can't see how to do that at the moment, so any help with that would be good)

The query below uses the following tables, simplified for this question

TABLE tbl_member 
  ( member_id INT AUTO_INCREMENT, -- pk
    member_sn` varchar(50) , --surname
    <other stuff>
  )

TABLE tbl_tenure
  (tenure_id INT AUTO_INCREMENT, -- pk
   member_id INT -- fk to tbl_member
   position_id -- fk to table of position titles
   date_started DATE
   date_ended DATE -- will be NULL if still in post
  )


  -- test data for query
  SET @the_test_tenure_start_date = '2016-05-13' ;
  SET @the_test_tenure_end_date = '2016-10-05';
  SET @the_test_position_id = 18; 
  SET @the_test_tenue_id = 122;

-- the query to return overlaps with data from a given tenure record  
SELECT
   tbl_tenure.tenure_id,
   tbl_tenure.member_id,
   tbl_member.member_sn,
   tbl_tenure.date_started,
   tbl_tenure.date_ended,
   CASE
      WHEN @the_test_tenure_end_date <= IFNULL(date_ended, CURDATE())  -- test end date <= existing end date
      AND @the_test_tenure_start_date >= date_started   -- test start date >= existing start date
      THEN 'Test dates fall completely inside an existing tenure'

      WHEN @the_test_tenure_end_date >= IFNULL(date_ended, CURDATE())   -- test end date >= existing end date
      AND @the_test_tenure_start_date <= date_started   -- test start date <= existing start date
      THEN 'An existing tenure falls completely inside test dates'

      WHEN @the_test_tenure_start_date >= date_started   -- test start date >= existing start date
      AND @the_test_tenure_start_date <= IFNULL(date_ended, CURDATE())   -- test start date <= existing end date
      THEN 'Test start date overlaps with an existing tenure'

      WHEN @the_test_tenure_end_date >= date_started   -- test end date >= existing start date
      AND @the_test_tenure_end_date <= IFNULL(date_ended, CURDATE())   -- test end date <= existing end date
      THEN 'Test end date overlaps with an existing tenure'
   END AS reason

FROM
   tbl_tenure
   INNER JOIN tbl_member
      ON tbl_tenure.member_id = tbl_member.member_id

WHERE ( -- there is an overlap (see qry 2.2 http://salman-w.blogspot.co.uk/2012/06/sql-query-overlapping-date-ranges.html
          @the_test_tenure_end_date >= date_started)    
          AND 
      IFNULL(date_ended, CURDATE()) >= @the_test_tenure_start_date
      )   
   AND tbl_tenure.position_id = @the_test_position_id   -- position to be tested
   AND  tbl_tenure.tenure_id <> @the_test_tenue_id    -- don't look at the test tenure record
ORDER BY tbl_tenure.date_started ASC;

In order to clarify this question, the output I am looking for is something like this, note tenure_id 132 where a member is recorded as ovelapping with themself

tenure_id   | member_id     | position_id   | start_date | end_date   | overlapping_member_id | overlapping_tenure_id | overlapping_start_date |overlapping_end_date | overlap_reason
   123      |      2        |      6        | 2016-02-01 | 2016 02-01 |       7               |       456             |     2016-01-05         |   2016-01-10         |'Test start date overlaps with an existing tenure'
   125      |      2        |      8        | 2016-02-01 | 2016 03-01 |       8               |       459             |     2016-01-0          |   2016-02-01         |'Test end date overlaps with an existing tenure'
   129      |      4        |      7        | 2016-03-10 | 2016 04-01 |       6               |       501             |     2016-03-2          |   2016-03-25         |'An existing tenure falls completely inside test dates'
   132      |      4        |      7        | 2016-01-01 | 2016 04-01 |       4               |       505             |     2016-03-01         |   2016-04-01         |'Test end date overlaps with an existing tenure'
   135      |      9        |      3        | 2016-05-01 | 2016 07-01 |       9               |       520             |     2016-04-0          |   2016-08-01         |'Test dates fall completely inside an existing tenure'
user3209752
  • 619
  • 2
  • 17
  • 29
  • See http://meta.stackoverflow.com/questions/333952/why-should-i-provide-an-mcve-for-what-seems-to-me-to-be-a-very-simple-sql-query. Note that Event B can be said to overlap Event A if Event B starts before Event A ends and ends after Event A starts. – Strawberry Oct 07 '16 at 11:46
  • But that does not distinguish between the case when B is completely enclosed within A or B extends beyond the end of A. Cases 1 and 3 in my case statement do that and provide a different reason. Besides my question is about using this query in a self join, not about detecting overlaps.As for the MCVE, I'm not sure I understand your point. I thought I gave a clear question with the table schema and an example query that others can benefit from. – user3209752 Oct 07 '16 at 12:15
  • Yes it does. It accounts for ALL cases – Strawberry Oct 07 '16 at 12:23
  • I agree, that's my point. If A goes from 2 - 8. Then if B goes from 4 - 6, 4 - 10, 1 - 10 or 6-10 they will ALL be detected by the condition you propose. I have no problem with that. However that condition does not distinquish between, eg B overlapping because B(4-6) is completely enclosed within A(2 - 8) and when B overlaps because B(4-10) has it's start within A's range but not it's end. I needed to distinquish between all four possible overlapping situations eg B(1-4), B(3-10), B(1-10) & B(4-6) in order to give the correct reason. But, as I say its the self join I would like help with. – user3209752 Oct 07 '16 at 13:11
  • Incidentally, would the person who downvoted the upvote I received please explain why they did so, in order that I can avoid making the same mistake in future – user3209752 Oct 07 '16 at 13:15

1 Answers1

0

After spending the day on it and having a think while running on my treadmill I believe I have the answer. I'm posting it here for the benefit of others. I moved the join with tbl_member into a sub query, and for the sake of completeness include another subquery to get the actual title of the position from a third table, tbl_position, shown below. (Can't see a way to replace the sub queries with joins but that doesn't matter.)

TABLE tbl_positions
   (
   position_id INT AUTO_INCREMENT, -- pk
   position VARCHAR(100), -- title of position
   <other stuff>
   )

The code I came up with is below which appears to work correctly and shows all the overlaps with details of who is overlapped with whom, when and why.

The oly niggle is that if, for example, Fred is shown to overlap as President with Jim's existing record with the reason that Fred's tenure completely encloses Jim's tenure, then Jim's position as President is also shown to overlap with Fred's existing tenure record with the reason given that Fred's is completely enclosed by Jim's. ie I get both sides of the overlap.

If there is a quick way to get just a 'one way' overlap then by all means post a better answer.

My answer

SELECT
   base_tenure.position_id     AS base_tenure_id,
   base_tenure.member_id     AS base_member_id,
   (SELECT member_sn FROM tbl_member WHERE tbl_member.member_id = base_tenure.member_id)     AS base_sn,
   (SELECT tbl_positions.position FROM tbl_positions WHERE tbl_positions.position_id = base_tenure.position_id ) AS POSITION,
   base_tenure.date_started  AS base_date_started,
   base_tenure.date_ended    AS base_date_ended,

   overlap_tenure.position_id  AS overlap_tenure_id,
   overlap_tenure.member_id AS overlap_member_id,
   (SELECT member_sn FROM tbl_member WHERE tbl_member.member_id = overlap_tenure.member_id)     AS overlap_sn,
   overlap_tenure.date_started AS overlap_date_started,
   overlap_tenure.date_ended   AS overlap_date_ended,  

   CASE
      WHEN base_tenure.date_ended <= IFNULL(overlap_tenure.date_ended, CURDATE())-- test end date <= existing end date
      AND base_tenure.date_started >= overlap_tenure.date_started  -- test start date >= existing start date
      THEN 'tbl_member dates fall completely inside an existing tenue'

      WHEN base_tenure.date_ended >= IFNULL(overlap_tenure.date_ended, CURDATE()) -- test end date >= existing end date
      AND base_tenure.date_started <=  overlap_tenure.date_started -- test start date <= existing start date
      THEN 'An existing tenue falls completely inside tbl_member dates'

      WHEN base_tenure.date_started >= overlap_tenure.date_started  -- test start date >= existing start date
      AND base_tenure.date_started <= IFNULL( overlap_tenure.date_ended , CURDATE()) -- test start date <= existing end date
      THEN 'tbl_member start date overlaps with an existing tenue'

      WHEN base_tenure.date_ended >= overlap_tenure.date_started  -- test end date >= existing start date
      AND base_tenure.date_ended <= IFNULL( overlap_tenure.date_ended , CURDATE())-- test end date <= existing end date
      THEN 'tbl_member end date overlaps with an existing tenue'
   END AS reason

FROM -- a self join on tbl_tenure
   tbl_tenure AS base_tenure,
   tbl_tenure AS overlap_tenure

WHERE (-- there is an overlap (see qry 2.2 http://salman-w.blogspot.co.uk/2012/06/sql-query-overlapping-date-ranges.html
           base_tenure.date_ended  >= overlap_tenure.date_started -- test end date >= existing start date
          AND 
          IFNULL(overlap_tenure.date_ended, CURDATE()) >= base_tenure.date_started 
          )

   AND 
      base_tenure.club_function_id = overlap_tenure.club_function_id -- positions are the same for both members

   AND  
      base_tenure.position_id <> overlap_tenure.position_id --  don't compare the base record with itself as they are identical and will always overlap 

 ORDER BY
       (SELECT member_sn FROM tbl_member WHERE tbl_member.member_id = base_tenure.member_id) ,
       base_tenure.date_started ;  
user3209752
  • 619
  • 2
  • 17
  • 29