Friday, February 15, 2013

new MaxL "export outline" XML to delimited text parser - orders of magnitude faster

The Java code in my last post works fine for XML containing less than 70,000 members, which is usually sufficient for BSO cubes. ASO cubes, however, can contain a LOT more members. Dan Pressman contacted me earlier in the week to let me know that his XML dimension export file containing 4 million members in almost 90 million lines of XML was taking over 17 hours to run (when it wasn't crashing from running out of memory).

So I rewrote the code to make it more efficient. The same XML above with 90 million lines now takes about 3.5 minutes to run and only uses 75MB of RAM. The functionality is even improved, in that member attributes are now also output. A header line is also prepended, which, due to Java I/O, adds about a minute to the process.

I recommend using this new code for all files instead of the code in the previous post. You can find the two files needed in my GitHub essbase-parse-export-outline-xml-to-text repository.

I also created a very simple Java GUI that allows you to choose the input file, output file, and delimiter:

You can use the XMLOutlineParser.jar file from the above GitHub repository to launch the GUI. If you have a fairly recent Java jre or jdk installed (version 6 or higher), you should be able to save the jar file to your local drive, then double-click on it to launch the GUI.

P.S. As you can see in the comments, some people have experienced issues when trying to save the output file on Windows. The following steps are the workaround to this permissions issue:
  1. Go to Start -> Programs -> Accessories
  2. right click on Command Prompt and select "Run as administrator"
  3. cd to the directory where you saved essbase-parse-export-outline-xml-to-text
  4. run "java -jar XMLOutlineParser.jar"
You can avoid this issue entirely by not selecting an output location at the root of the C directory (e.g. C:\test). Rather use one such as "C:\Documents and Settings\harry\outputFile.txt".

Update: I've added the ability to pass in command line arguments, making XMLOutlineParser suitable for scripting in batch and shell scripts. Passing in arguments will prevent the GUI from running. 
  • run java -jar XMLOutlineParser.jar "input file: path & name" "output file: path & name" "field separator to use in output file"
  • -------e.g. java -jar XMLOutlineParser.jar "/Users/harry/year.xml" "/Users/harry/Documents/year.txt" "!"
Posted by at
Labels: , ,

5 comments:

  1. Gary Crisci, Oracle Ace DirectorFebruary 11, 2014 at 1:21 PM

    Hi - I exported the Year dim from Sample Basic. I dropped it in C:\Dev. I run the essbase-parse-export-outline-xml-to-text.jar, put in the source, target and delimiter and hit Run.
    A box pops up immediately that says "Elapsed Time. time to run:00:00:00" with an OK button. If I click OK the box closes and no txt file is generated. I have left it open for a while to see if it just needs time, but nothing happens.
    Any suggestions?

    ReplyDelete
    Replies
    1. Harry GatesFebruary 11, 2014 at 1:37 PM

      Hi Gary,

      I'm by no means a Windows expert, but I've seen strange permission issues since Windows 7 for folders right under the C drive. Can you try to select a different output location - maybe under "My Documents"?

      Delete
    2. Gary Crisci, Oracle Ace DirectorFebruary 11, 2014 at 3:55 PM

      Thanks, I tried various combinations of locations for source and target as well as where I ran the .jar from, nothing seems to work. I am an admin on the machine so permissions shouldn't be an issue. Disappointed, this looked like a cool utility, would love to figure out a way to use it.

      Delete
    3. Harry GatesFebruary 11, 2014 at 5:28 PM

      If you run the jar from the command line, it will give you more information about what's happening. Try "java -jar essbase-parse-export-outline-xml-to-text.jar".

      I also have admin rights on my Windows machine, yet when I run the above and select C:\test\year.txt as the output file, I see the following in the command line in which I ran the java jar: "C:\test\year.txt (Access is denied)"

      Delete
    4. Harry GatesFebruary 11, 2014 at 5:38 PM

      The following appears to be a valid workaround for the Windows permission issue:
      1) Start -> Programs -> Accessories
      2) right click on Command Prompt and select "Run as administrator"
      3) cd to the directory where you saved essbase-parse-export-outline-xml-to-text
      4) run "java -jar essbase-parse-export-outline-xml-to-text.jar"

      Delete

Subscribe to: Post Comments (Atom)