Input-output through stream - Longchar Input

Question

help with the following would be appreciated.

I am trying to run an OS-command in progress using input-output through.

Case:

Run command: "echo 'this is a long string'"
Capture output: "'this is a long string'"

Issues are:

My input string may be greater than the character limit
When running directly using through "program" or through value("expression") The input is truncated to ~2500-3000 chars.
Entire operation runs fine using files, but want to avoid IO and management.
Having trouble attaching my data to the OS-command (utilizing put OR export)

Could anyone suggest a method to execute a command line, with attached data exceeding 35,000 characters?

This is what I have tried: (Along with many permutations)

define variable cLongMsg as character  no-undo.
define variable iLoop as integer no-undo.
define variable lcResp as longchar no-undo.
define variable cFileLine as character no-undo.    
define stream logStream.

input-output stream logStream through "curl".
  /* execute curl -- attach data using put */
  put stream logStream "http://stackoverflow.com".
  put stream logStream "-d 'key=long_data"
  put stream logStream "&value=more_long_data'"
  put stream logStream CONTROL NULL(0).
output close.     

read_loop:
repeat:

    import stream logStream unformatted cFileLine.
    assign lcResp = lcResp + cFileLine.
end.

input close.
input-output close.

message "[OUT] " + string(lcResp) view-as alert-box.

Currently used solution:

Progress File

function CURL_long returns longchar private (
         input cInput as longchar).

  def var cUnixCMD                      as longchar                     no-undo.
  def var cDataPart                     as char                         no-undo.
  def var cData                         as longchar                     no-undo.    
  def var cResultChunk                  as char                         no-undo.
  def var iResultChunkMax               as inte      init 30000         no-undo.
  def var iUsedChunkMax                 as inte                         no-undo.
  def var iResultPos                    as inte                         no-undo.

  fix-codepage(cData) = "UTF-8".

  assign cUnixCMD = "curl " + cInput + " | fold -c30000".

  input-output stream CURL_CMD through "input_buffer.sh".

  /* set the result position */
  assign iResultPos = 0.

  ResultLoop:
  repeat:
    if iResultPos + iUsedChunkMax > length(cUnixCMD) then
    do:
      assign cResultChunk = substring(cUnixCMD, iResultPos + 1).
      put stream CURL_CMD unformatted cResultChunk skip.
      leave ResultLoop.
    end.

    /* set the current chunk of the result */
    assign cResultChunk = substring(cUnixCMD, iResultPos + 1, iResultChunkMax)
           iUsedChunkMax = iResultChunkMax.

    put stream CURL_CMD unformatted cResultChunk skip.

    assign iResultPos = iResultPos + iUsedChunkMax.

  end. /* ResultLoop: repeat: */
  output stream CURL_CMD close.

  read_loop:
  repeat:
    import stream CURL_CMD unformatted cDataPart.
    if cDataPart <> "" then assign cData = cData + cDataPart.
    else leave.
  end.

  return cData.

end function.

Bash File

#!/bin/bash
myCMD=""
while IFS= read line
do
  myCMD+=$line
done    
eval $myCMD
exit

For now I will leave the input limitation at bay and treat the question as answered. We were able to avoid file IO and utilize longchar data through a third-party shell script. Where it executes the shell, and repeatedly accepts input, buffering the data -then ending once a control character is received. — Screll, Jan 23 '19 at 17:14

Tom Bascom · Accepted Answer · 2019-01-21T18:41:06.780

You're on the right track with a few small syntax issues.

This works for me:

define variable lcResp as longchar no-undo.
define variable cFileLine as character no-undo.

define stream logStream.

input stream logStream through "curl https://stackoverflow.com".

read_loop:
repeat:

  import stream logStream unformatted cFileLine.

  assign lcResp = lcResp + cFileLine + "~n".

end.

input stream logStream close.

message "[OUT] " length( lcResp ) view-as alert-box.

COPY-LOB lcResp to file "zzz".

The MESSAGE statement cannot output lcResp when it exceeds roughly 30,000 characters. I used length( lcResp ) to show that I got the data.

Output statements such as MESSAGE, PUT or EXPORT do not work with longchar data. You need to use COPY-LOB to directly read or write a longchar.

Also -- be careful with the handling of newlines or the lack thereof. INPUT that isn't terminated with a newline will be lost by IMPORT in a REPEAT loop. I usually use something more like this to avoid that:

define variable lcResp as longchar no-undo.
define variable cFileLine as character no-undo.

define stream logStream.

input stream logStream through "curl https://stackoverflow.com".

read_loop: do while true:

  cFileLine = ?.

  do on endkey undo, leave
     on error undo, leave:

    import stream logStream unformatted cFileLine.

  end.

  if cFileLine = ? then
    leave read_loop.
   else
    lcResp = lcResp + cFileLine + chr(13) + chr(10).

end.

input stream logStream close.

message "[OUT] " length( lcResp ) view-as alert-box.

COPY-LOB lcResp to file "zzz".

The "chr(13) + chr(10)" business might be important if your newlines need to be a specific format.

Thank you very much Tom Bascom, you gave some very informative pointers. I will especially take your input examples into account. I will post our current workaround below. — Screll, Jan 23 '19 at 16:56

score 0 · Answer 2 · answered Jan 21 '19 at 15:25

0

You can use -inp startup parameter to overcome the problem. More info here: https://documentation.progress.com/output/ua/OpenEdge_latest/index.html#page/dpspr/input-characters-(-inp).html

answered Jan 21 '19 at 15:25

Austin

1,237
1
11
22

The -inp option impacts compiling code -- not the length of input data. – Tom Bascom Jan 21 '19 at 15:58

Input-output through stream - Longchar Input

2 Answers2