Intellij uses wrong line numbers for de-lombok source

Question

I followed the instructions in Create and install de-lomboked source jar in maven for delomboking my source code and including it in the jar. So now, in another project, I can access the delomboked source. The problem is that IntelliJ uses this source to display when I click on a stack trace and the line numbers are wrong. IntelliJ is using the line numbers from the compiled source and it doesn't match up exactly with the actual source. Is there around this? If not, I'll have to go back to using the regular source code, instead of de-lomboked