Currently you can only access on premise SQL Server databases by building a model & publishing it to powerbi.com using the Power BI Desktop.
If you select "Import" as your connection option, Power BI will store the required data online. You will be able to query on that fixed data and work with it even when your connection to the on premises server is down. You can update your fixed dataset with new data if your connection is properly defined in your gateway.
If you select "DirectQuery" in your Power BI Desktop you will require an active connection to the On premises server for any reporting to work online. On the up side, it'll work on live data and will not store any data online.
When publishing, Power BI Desktop will try to match your connections used in your report to the ones defined in your gateways. If it can find a match, it'll automaticly link those up. It'll give you an informational message to let you know if this step succeeded after publishing. At the same time it'll add a dataset using the same data you used in the reports you just built.