Yes, you can.
The document you shared is Microsoft's official and therefore there is no doubt that you can upgrade it to any higher version using dump and restore.
Just take care of the below mentioned points:
You can upgrade your PostgreSQL server deployed in Azure Database for PostgreSQL by migrating your databases to a higher major version server using following methods.
- Offline method using PostgreSQL
pg_dump
and pg_restore
which incurs downtime for migrating the data.
- Online method using Database Migration Service (DMS). This method provides a reduced downtime migration and keeps the target database in-sync with the source and you can choose when to cut-over. However, there are few prerequisites and restrictions to be addressed for using DMS.
The following table provides some recommendations based on database sizes and scenarios.

Choose the right approach based on your database configuration and it should be done without any issue.
To upgrade using pg_dump
and pg_restore
, you can refer Migrate your PostgreSQL database by using dump and restore.