0

I am trying to run gdal_merge using BIGTIFF=YES from python.

command = "gdal_merge.py -co BIGTIFF=YES -o " + output_path + "/merged.tif" + " -of gtiff " + merge_string
print(os.popen(command).read())

for example this kind of command will be executed.

gdal_merge.py -co BIGTIFF=YES -o /root/data/sentinel_merged_caculated/20180116/B03/merged.tif -of gtiff /root/data/sentinel_2/S2A_MSIL1C_20180116T082251_N0206_R121_T35MNS_20180116T120855.SAFE/GRANULE/L1C_T35MNS_A013417_20180116T083253/IMG_DATA/T35MNS_20180116T082251_B03.tif /root/data/sentinel_2/S2A_MSIL1C_20180116T082251_N0206_R121_T35MMT_20180116T120855.SAFE/GRANULE/L1C_T35MMT_A013417_20180116T083253/IMG_DATA/T35MMT_20180116T082251_B03.tif /root/data/sentinel_2/S2A_MSIL1C_20180116T082251_N0206_R121_T35MNR_20180116T120855.SAFE/GRANULE/L1C_T35MNR_A013417_20180116T083253/IMG_DATA/T35MNR_20180116T082251_B03.tif /root/data/sentinel_2/S2A_MSIL1C_20180116T082251_N0206_R121_T35MPT_20180116T120855.SAFE/GRANULE/L1C_T35MPT_A013417_20180116T083253/IMG_DATA/T35MPT_20180116T082251_B03.tif /root/data/sentinel_2/S2A_MSIL1C_20180116T082251_N0206_R121_T35MNT_20180116T120855.SAFE/GRANULE/L1C_T35MNT_A013417_20180116T083253/IMG_DATA/T35MNT_20180116T082251_B03.tif /root/data/sentinel_2/S2A_MSIL1C_20180116T082251_N0206_R121_T35MPS_20180116T120855.SAFE/GRANULE/L1C_T35MPS_A013417_20180116T083253/IMG_DATA/T35MPS_20180116T082251_B03.tif /root/data/sentinel_2/S2A_MSIL1C_20180116T082251_N0206_R121_T35MPU_20180116T120855.SAFE/GRANULE/L1C_T35MPU_A013417_20180116T083253/IMG_DATA/T35MPU_20180116T082251_B03.tif /root/data/sentinel_2/S2A_MSIL1C_20180116T082251_N0206_R121_T35MNU_20180116T120855.SAFE/GRANULE/L1C_T35MNU_A013417_20180116T083253/IMG_DATA/T35MNU_20180116T082251_B03.tif

But in the output it shows

ERROR 1: TIFFAppendToStrip:Maximum TIFF file size exceeded. Use BIGTIFF=YES creation option.
0...10...20...30...40...50...60...70...80...90...100 - done.

and the some picture will not be merged.

Can anyone tell me what's happening and how to allow BIGTIFF?

The info of the device and sowtware I'm using is as below

root@3a441aa2d7df:~/opt# gdalinfo --version
GDAL 3.3.2, released 2021/09/01

root@3a441aa2d7df:~/opt# cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.4 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.4 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal

root@3a441aa2d7df:~/opt# python3 --version
Python 3.8.10
Yunnosch
  • 26,130
  • 9
  • 42
  • 54
komura_
  • 9
  • 2
  • Maybe try with `gdal_merge.py --debug ON ...` – Mark Setchell Aug 27 '22 at 07:28
  • This is probably either Python or gdal related problem. Print the `command`. Is it correct? Does it run fine from the shell? – VPfB Aug 27 '22 at 07:49
  • You could also hexdump the first 4 bytes of any TIFF output file it produces and see if the version number is 42 (regular TIFF) or 43 (BIGTIFF). If you don't have tool to do hexdump, use https://hexed.it/ online. – Mark Setchell Aug 27 '22 at 09:24
  • @MarkSetchell I have debugged and I only see each of the image being GDALOPEN and GDALCLOSE and there is no error execpt the `GDAL: Flushing dirty blocks: 0...10...20...30...40...50...60...ERROR 1: TIFFAppendToStrip:Maximum TIFF file size exceeded. Use BIGTIFF=YES creation option.` I did the hexdump comannd and it shows as below. it's not 42 or 43. what does this mean? `root@3a441aa2d7df:~/data/sentinel_merged_caculated/20180116/B03# hexdump -n4 merged.tif 0000000 4949 002a 0000004 ` – komura_ Aug 29 '22 at 05:55
  • @VPfB I did it directly from shell as well, but it only shows the same output – komura_ Aug 29 '22 at 05:57
  • The `49 49` in your hexdump is `II` for Intel byte ordering and `00 2a` is `42` - meaning it is definitely not writing a BIGTIFF, just a regular TIFF. I don't know what else to suggest, sorry. Have you got newest versions of everything? – Mark Setchell Aug 29 '22 at 06:05
  • @幸村龍之介 Thanks for doing the test. In this case I suggest to delete the [Python] tag. – VPfB Aug 29 '22 at 06:31
  • @MarkSetchell Thanks for the tips. I found out that the gdal_merge gets an error if there is the same output file name already and does not overwrite the file. – komura_ Aug 30 '22 at 06:18
  • @VPfB Thanks for the tips. I found out that the gdal_merge gets an error if there is the same output file name already and does not overwrite the file – komura_ Aug 30 '22 at 06:18

1 Answers1

1

I found out that the gdal_merge gets an error if there is the same output file name already and does not overwrite the file

komura_
  • 9
  • 2