-2

This is more of a conceptual question as there is no relevant documentation available. We have an on prem IBM-MQ from which we need to transfer data on our cloud storage bucket (GCP/AWS), what could be possible solutions in this case? Any help or direction would be appreciated. Thank you!

Sam
  • 161
  • 1
  • 9
  • Do you want to connect an on-prem MQ Queue Manager to a cloud hosted MQ Queue Manager? – Morag Hughson Feb 01 '22 at 02:43
  • @MoragHughson so on-prem MQ Queue Manager to cloud native (Bucket or Pub Sub) – Sam Feb 01 '22 at 07:50
  • You can't transfer data file using the Google Cloud Storage bucket based on this [answer](https://stackoverflow.com/questions/40547444/google-pubsub-to-google-cloud-storage). It's recommended to use [Cloud Storage FUSE](https://cloud.google.com/storage/docs/gcs-fuse?hl=en) / [FIleStore](https://cloud.google.com/filestore). Possible to manage once MQ has been set up on Compute Engine and connected to the Cloud Storage FUSE/FIle Store. May we know what will be the purpose of the storage once the MQ-Data has been transferred? – JaysonM Feb 01 '22 at 08:09
  • 1
    @JaysonM I get your point about the google cloud storage bucket. The main goal is to get the Data into BigQuery to do analytics (might not need storage at all). We already have a VPN connection established to the on-prem system. IBM MQ is running on-prem. – Sam Feb 01 '22 at 08:38

1 Answers1

0

I'm assuming you can reach your goal once the MQ-data has been changed/converted to supported format by the Big Query.

You can refer on this google documentation for full guide on Loading data from local files. You can upload file via GCP Console or using selected programming language that will match on your on-prem. There's also variety of uploads that you can choose from according to data file. This also includes the right permission to use the BigQuery.

If you require authentication you check on this Big Query Authentication Guide

JaysonM
  • 596
  • 1
  • 10
  • 1
    How does IBM-MQ fit into this solution? – JoshMc Feb 02 '22 at 04:20
  • As per my understanding on OP's comment MQ-Data needs to transfer Data on Big Query to perform analytics. I'm just providing my possible resolution based on OP's comment. – JaysonM Feb 02 '22 at 04:25
  • @JaysonM Thanks for your effort, but the problem with your proposed solution is how do we get the connection from IBM-MQ to GCP ? We can not upload files through console in this case – Sam Feb 03 '22 at 11:28