0

Before I begin, yes I know everything I am about to explain is back-asswards but at the moment it is what I have to work with.

My organization is in the middle of a Cognos 10 BI implementation, at the moment we are having HUGE performance issues with the data cubes, severely hampering our end users ability to slice data in an ad-hoc fashion. Historically we used large data extracts from SAP, manipulated in ms-access to provided a data source that was updated daily that end users could Pivot around in Excel.

As this is NOT transactional data, it worked as we never had more than a half million records, performance was never an issue.

As our implementation team has been unable to provide management with functioning data cubes we can use to provide static views and reports I have been tasked with using Cognos data extracts to re-create the old system temporarily.

The issue I am running into is that randomly, 3 times a week, 1 time the next, the files will contain unparsable records. I doubt it is a special character issue as I can re-download the file and it functions fine the 2nd or 3rd time.

Does anyone have any experience with something similar? I realize the data sets provided by Cognos were not designed for this purpose, but it seems strange that 20 percent of the files will contain corruptions. Also strange is that when I select a .xls spreadsheet as the download format, it seems to be a Unicode text file with the extension changed to .xls

Any insight would be appreciated.

EDIT: Diffing the files will be my next experiment, even though they are byte for byte comparable, I HAVE however compared the specific records that are unparsable in one file, yet parsable in the next and have not found any difference.

As for the import, I manually convert the file to a Unicode text and import it from that format.

pegicity
  • 359
  • 1
  • 3
  • 12
  • 1
    In what format are these data extracts? CSV files, perhaps? When you re-download a file that previously failed have you checked to see if it two files are exactly the same? (or, if not byte-for-byte identical have you tried to [diff](http://en.wikipedia.org/wiki/Diff_utility) the two files?) Also, how are you importing them? Please [edit] your question with details. – Gord Thompson Feb 05 '15 at 16:21
  • Hey Gord, updated a few things for you, as far as the format it is output in a ".xls" spreadsheet format, though excel will warn me that the format does not match the extension and defaults to Unicode text if you attempt to save the file. CSV is an option I have not explored yet, would you recommend? – pegicity Feb 05 '15 at 17:04
  • When I wrote "byte-for-byte identical" I was really thinking of `fc /b` from a command line, not just the exact same size. – Gord Thompson Feb 05 '15 at 17:31

0 Answers0