Check for error messages in the Tableau log here: My Documents > My Tableau Repository > Logs > log.txt
In our case we saw Tableau appended $inlinecount=allpages to the OData URL, which fails in OData v4 with:
{"error":{"code":null,"message":"The system query option
'$inlinecount' is not defined."}}
Tableau only supports OData v2. Denodo can support both versions. We installed v2 and we were able to connect successfully. Here is feedback from Tableau Support:
According to this article from Microsoft, the inlinecount function was
not included in OData version 4:
https://msdn.microsoft.com/en-us/library/dd942040.aspx. This is one of
the reasons why Tableau Desktop cannot connect to that version of
OData.
The only Tableau documentation I could find documenting the OData V2 requirement was this:
Tableau connects to OData V2, and does not support browsing OData
service documents.
http://onlinehelp.tableau.com/current/pro/online/windows/en-us/help.htm#examples_odata.html
We originally were concerned about json vs. atom formats based on other answers here, but this was not issue for us. Once we got the connection working, we realized that Tableau cannot use a live connection to OData so we reverted back to ODBC anyways!
TL;DR Make sure you are using OData V2 and not using an OData service document (as of August 2016, Tableau v 9.3.5 and below).