I am currently making a machine learning model to predict the Remaining Useful Lifetime (RUL) of Turbofan engines. I was not happy with my previous noise defiltering method. So I decided to try a Savgol filter. I implemented the filter with (enough) succes, however I cannot find a way to improve the edges of my Savgol filter. The filter denoises the signal, however at the end the signal drops off. Is there a way to somehow improve the last 10% of the signal?
I tried changing the mode of the savgol filter, however there was no succes. The goal I expect to achieve is a reasonably denoised signal without any unnecessary dropoffs, especially around the edges.
This is my code:
df1 = pd.DataFrame(df1)
df1_unsmoothed = deepcopy(df1)
#preparing to filter the noisy signal, while keeping the first two columns unfiltered in the dataframe
df1_first_two_cols = df1.iloc[:, :2]
smooth_data = df1.iloc[:,2:].apply(lambda x: savgol_filter(x, window_length=31, polyorder=2, mode='nearest'))
smooth_data = pd.concat([df1_first_two_cols, smooth_data], axis=1)
smooth_data.columns = df1.columns
df1 = pd.DataFrame(smooth_data)
#plotting
raw_signal = df1_unsmoothed.loc[df1_unsmoothed.Equipment == 1, str(2)]
denoised_signal = smooth_data.loc[smooth_data.Equipment == 1, str(2)]
plt.plot(raw_signal)
plt.plot(denoised_signal)
plt.show()
Is there a way to improve the denoising around the edge?
