How to split a string with a multi-character delimiter in InstallScript? (StrGetTokens fails)

Question

The InstallScript function StrGetTokens allows you split a string into a list. See: http://helpnet.flexerasoftware.com/installshield20helplib/Subsystems/installshield20langref/Content/helplibrary/LangrefStrGetTokens.htm

This works great with single character delimiter. Further, you may specify a "delimiter set", i.e. a list of 1 character delimiters which may all be employed. There is no apparent way to specify a multi-character delimiter. I tried before realizing the string I specified would treated as this "delimiter set" instead of the delimiter itself, and ran into very unexpected results. Must I roll my own tokenizer, or import one, etc.?

BuvinJ · Answer 1 · 2015-12-23T18:02:10.697

I rolled by own in InstallScript. You can directly substitute a call to StrGetTokens with this function. It just treats the "delimiter set" parameter as a multi-character delimiter instead. I wish there was a better solution...

//prototype NUMBER Tokenize( BYREF LIST, STRING, STRING );
//---------------------------------------------------------------------------
//                                                                           
// Function:  Tokenize
//                                                                           
// Purpose:   Use in place of StrGetTokens when your delimiter is more than 
//            1 character in length
//
// Returns: tokenList is modifed by referrence
//      = 0     Tokenized
//      < 0 Failed to tokenize
//---------------------------------------------------------------------------
function NUMBER Tokenize( tokenList, szDelimitedValues, szDelimiter )
    STRING szReplacementToken, szReplacementTokenPlaceHolder;
    STRING svToken;
    NUMBER nListResult;
begin

    // Clear the return list
    ListDeleteAll( tokenList );

    // Return failure if attempting to tokenize an empty string
    if( szDelimitedValues = "" ) then
        return -1;
    endif;

    // When the delimiter is 0 or 1 chars, just use the native function
    if( StrLength( szDelimiter ) < 2 ) then
        return StrGetTokens( tokenList, szDelimitedValues, szDelimiter );
    endif;

    // Define a 1 character replacement delimiter, and a "placeholder" for it
    szReplacementToken            = "~";
    szReplacementTokenPlaceHolder = "[__TOKEN_PLACEHOLDER__]";

    // Replace all instances of the replacement token with its placeholder
    StrReplace( szDelimitedValues, szReplacementToken, szReplacementTokenPlaceHolder, 0 );

    // Replace all instances of the "real" delimiter with its replacment
    StrReplace( szDelimitedValues, szDelimiter, szReplacementToken, 0 );

    // Tokenize, using the replacement delimiter
    if( StrGetTokens( tokenList, szDelimitedValues, szReplacementToken ) < 0 ) then
        // If this fails, clear the tokenList and return failure
        ListDeleteAll( tokenList );
        return -1;
    endif;

    // Iterate through the token list
    nListResult = ListGetFirstString( tokenList, svToken ); 
    while( nListResult != END_OF_LIST )             

        // For each token,
        // replace all instances of the "placeholder" with the delimiter employed
        // i.e. restore the original value
        StrReplace( svToken, szReplacementTokenPlaceHolder, szReplacementToken, 0 );

        // Update the token in the list with this restored version of it
        if( ListSetCurrentString( tokenList, svToken ) < 0 ) then
            // If this fails, clear the tokenList and return failure
            ListDeleteAll( tokenList );
            return -1;
        endif;      

        nListResult = ListGetNextString( tokenList, svToken ); 
    endwhile; 

end;

How to split a string with a multi-character delimiter in InstallScript? (StrGetTokens fails)

1 Answers1