mplfinance xaxis ticks every n minutes, but not if datapoint doesn't exist

Question

Checking I am not re-inventing the wheel here, with mplfinance I would like to have ticks on the x-axis every 15 minutes, but only if data exists.

Plotting direct to mplfinance (without returnfig=True) the plot is ok except the xaxis which values are not time aligned they depend on the first element of the dataframe as to what time gets used.

To try and make this have grid/tick every hour, I have the below code which works ok when there are the expected panda indexes in my dataset:

start_date = pd.to_datetime('2021-12-21 04:00').tz_localize('America/New_York')
end_date = pd.to_datetime('2021-12-21 19:55').tz_localize('America/New_York')
df5trimmed = df5.truncate(before=start_date, after=end_date)    
                    
ticks = pd.date_range(start_date, end_date, freq='15T')
ticklocations = [ df5trimmed.index.get_loc(tick) for tick in ticks ]
ticklabels = [ tick.time().strftime('%H:%M') for tick in ticks ]

fig, axlist = mpf.plot(df5trimmed,style='yahoo', addplot=plotsToAdd, figsize=(48,24), 
                       type='candlestick', volume=True, xrotation=0,
                       tight_layout=True, returnfig=True)

axlist[-2].xaxis.set_ticks(ticklocations)
axlist[-2].set_xticklabels(ticklabels)

However it blows up, expectedly, with index not found during the iterator df5trimmed.index.get_loc(tick) for tick in ticks when no data exists in the pandaframe for that tick.

Notice the discontinuities in the data below it blows up during 17:00 key access attempt, as it doesn't exist in my data:

Essentially I am looking to plot the lines aligned to n minutes (in the example below 15 minutes), but only if it exists and not otherwise (if it doesn't exist, I am ok with the bars being right next to one another)... in summary during regulary trading hours with liqudity (where there would be data points) would be ticks @ 08:15, 08:30.

Is there an argument in mplfinance that can do this?

What I am looking to achieve

The below is from tradingview, note the aligned time ticks every 15 minutes during regular trading hours and pretty much the entire plot.

Additional Info - source data and what is plotted

The below uses this csv data, and plots directly to mplfinance, you can see the time ticks are not aligned to the hour I get 04:00, 06:25, 08:10, 09:50 etc:

import mplfinance as mpf
import pandas as pd
import datetime

df5trimmed = pd.read_csv('https://pastebin.com/raw/SgpargBb', index_col=0, parse_dates=True)
fig, axlist = mpf.plot(df5trimmed,style='yahoo', figsize=(48,24), type='candlestick', volume=True, xrotation=0, tight_layout=True, returnfig=True)

# Display:
mpf.show()

Answer 1

I used the following code to add the ticks, but only if they exist - I suspect much nicer ways of writing this so open to reposts of this as alternative answers.

My next iteration will be to make the text with 0-degree rotation, and not cluttered should a label be overwritten by another.

import mplfinance as mpf
import pandas as pd
import datetime

def getTimestampTickFrequency(df):
    # get most common interval in minutes
    mode = df.index.to_series().diff().astype('timedelta64[m]').astype('Int64').mode()[0]
    
    if mode==5:
        return 15, 3 # for 5 minutes, tick every 15 mins
    elif mode==15:
        return 60, 12 # for 15 minute data, tick every hour
    elif mode==120:
        return 240, 48 # for hourly data, tick every 2 hours
    
    return mode

def getTickLocationsAndLabels(df):
    tickLocations = []
    tickLabels = []
    
    tickFrequencyMinutes, samplesBetweenTicks = getTimestampTickFrequency(df)
    entireTickRange = pd.date_range(start=df.index[0], end=df.index[-1], freq=f'{tickFrequencyMinutes}T')
    prevTimestamp = df.index[0]
    
    # get indexes of data frame that match the ticks, if they exist
    for tick in entireTickRange:
        print(tick)
        try:
            found = df.index.get_loc(tick)
            currentTimestamp = df.index[found] 
            timestampDifference = (currentTimestamp - prevTimestamp).total_seconds() / 60
            
            print(f'Delta last time stamp = {timestampDifference}')
            #if timestampDifference <= tickFrequencyMinutes:
            tickLocations.append(found)
            tickLabels.append(tick.time().strftime('%H:%M'))
            
            prevTimestamp = currentTimestamp
        except KeyError:            
            pass # ignore
            
    return tickLocations, tickLabels

df5trimmed = pd.read_csv('https://pastebin.com/raw/SgpargBb', index_col=0, parse_dates=True)
tickLocations, tickLabels = getTickLocationsAndLabels(df5trimmed)
fig, axlist = mpf.plot(df5trimmed,style='yahoo', figsize=(48,24), type='candlestick', volume=True, tight_layout=True, returnfig=True)
axlist[-2].xaxis.set_ticks(tickLocations)
axlist[-2].set_xticklabels(tickLabels)

# Display:
mpf.show()

Answer 2

So, what need is to remove gaps from the plot due to missing bars.

Essentially I am looking to plot the lines aligned to n minutes (in the example below 15 minutes), but only if it exists and not otherwise

From the code you posted to try, but zooming in the first 30 bars and adding the option show_nontrading=True:

import mplfinance as mpf
import pandas as pd
import datetime

df5trimmed = pd.read_csv('https://pastebin.com/raw/SgpargBb', index_col=0, parse_dates=True)
fig, axlist = mpf.plot(df5trimmed[0:30],
                       style='yahoo', 
                       figsize=(48,24), 
                       type='candle', 
                       volume=True, 
                       xrotation=0, 
                       tight_layout=True, 
                       returnfig=True,
                       show_nontrading=True)

# Display:
mpf.show()

I get this, which is showing the gaps you mentioned.

But if I set the option show_nontrading=False

This changes to the plot below, which removes the gaps corresponding to the missing bars.

Isn't this what you needed?

Answer 3

Please check if this solves your issue. I think it does.

Check the date range between 6:00 am and 7:00 AM. Few bars are plotted between 5:00 and 6:00 and 6:30 is missing.

import mplfinance as mpf
import pandas as pd
import datetime
import matplotlib.pyplot as plt
%matplotlib inline

# DATA PREPARATION
df5 = pd.read_csv('https://pastebin.com/raw/SgpargBb', index_col=0, parse_dates=True)
start_date = pd.to_datetime('2021-12-21 1:00').tz_localize('America/New_York')
end_date = pd.to_datetime('2021-12-21 7:00').tz_localize('America/New_York')
df5trimmed = df5.truncate(before=start_date, after=end_date)

# PLOTTING 
fig,  axlist= mpf.plot(df5trimmed,
                       style='yahoo', 
                       type='candlestick', 
                       volume=True, 
                       xrotation=0, 
                       figsize=(20,10), 
                       returnfig=True,
                       show_nontrading=False)

# x-tick labeling preparation
# the frequency can be ajusted but needs to be lower than original data
idx = pd.date_range(start_date, end_date, freq='30T', tz='America/New_York')
df_label_idx = pd.DataFrame(index=idx)

# this merge does the trick: the output is the intersection between the lower frequency and the 
# higher freq time series. The inner option leaves in the output only those rows present in both TS
# dropping from the lower freq TS those missing periods in the higher freq TS.
df_label = pd.merge(df_label_idx, df5trimmed, how='inner', left_index=True, right_index=True ).tz_convert('America/New_York')

# Tick labels are generated based on df_label
tick_labels = list(df_label.index.strftime('%H:%M'))
ticklocations = [df5trimmed.index.get_loc(tick) for tick in df_label.index ]

axlist[-2].xaxis.set_ticks(ticklocations)
axlist[-2].set_xticklabels(tick_labels)

mpf.show()


print(tick_labels)

df_label

df5trimmed['2021-12-21 05:00:00-05:00':'2021-12-21 07:00:00-05:00']