1

I want to highlight my python log outputs with sublime syntax definitions according to my custom python logging formatter. I try below syntax definition but it did not work. I can't see this highlight on my sublime highlight list. Below is my python CustomFormatter class, sublime syntax definitions and example log. Can you give me any solution?

My syntax definition is:

%YAML 1.2
---
# See http://www.sublimetext.com/docs/syntax.html
name: TBLog
file_extensions:
  - tblog
scope: source.tblog
contexts:
  main:
    - match: '^\[.*\].*'
      captures:
        '0': { name: string.timestamp.tblog }
        '1': { name: string.loglevel.tblog }
      push: log_message

  log_message:
    - match: '$'
      pop: true
    - include: string.quoted.double
    - include: string.quoted.single
    - include: keyword.control
    - include: constant.numeric
    - include: string.timestamp

  string.timestamp:
    - match: '\[\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}\]'
      scope: string.timestamp.tblog

  string.quoted.double:
    - match: '"'
      scope: punctuation.definition.string.begin.tblog
      push:
        - meta_scope: string.quoted.double.tblog
        - match: '\\.'
          scope: constant.character.escape.tblog
        - match: '"'
          scope: punctuation.definition.string.end.tblog
          pop: true

  string.quoted.single:
    - match: "'"
      scope: punctuation.definition.string.begin.tblog
      push:
        - meta_scope: string.quoted.single.tblog
        - match: '\\.'
          scope: constant.character.escape.tblog
        - match: "'"
          scope: punctuation.definition.string.end.tblog
          pop: true

  keyword.control:
    - match: '\b(if|else|for|while)\b'
      scope: keyword.control.tblog

  constant.numeric:
    - match: '\b(-)?[0-9.]+\b'
      scope: constant.numeric.tblog

  string.loglevel:
    - match: '\[(WARNING|ERROR|CRITICAL|DEBUG|INFO)\]'
      captures:
        '0': { name: string.loglevel.tblog }
      scope: meta.loglevel.tblog
      push: log_message

  meta_scope:
    string.loglevel.tblog: 'meta.loglevel.tblog'

My custom python formatter is:

class ColorFormatter(logging.Formatter):
    COLORS = {
        'WARNING': colorama.Fore.YELLOW + colorama.Style.BRIGHT,
        'ERROR': colorama.Fore.RED + colorama.Style.BRIGHT,
        'CRITICAL': colorama.Fore.WHITE + colorama.Style.BRIGHT + colorama.Back.RED,
        'DEBUG': colorama.Fore.BLUE + colorama.Style.BRIGHT,
        'INFO': colorama.Fore.GREEN + colorama.Style.BRIGHT,
        'DEFAULT': colorama.Fore.WHITE,
    }

    def __init__(self, fmt="[%(asctime)s][%(levelname).1s]%(message)s", datefmt="%Y-%m-%d %H:%M:%S"):
        logging.Formatter.__init__(self, fmt, datefmt)

    def format(self, record):
        color = self.COLORS.get(record.levelname, self.COLORS['DEFAULT'])
        message = logging.Formatter.format(self, record)
        return f"{color}{message}\033[0m"

Example log is:

[2023-02-23 22:58:25,226][I]TC] Initializing the bot!
[2023-02-23 22:58:25,241][D]EX] Creating exchange client.
[2023-02-23 22:58:25,678][D]EX] Exchange client has been created!
[2023-02-23 22:58:25,709][I]TC] Preparing bot to start trading!
[2023-02-23 22:58:25,725][D]TC] Variables has been reset!
[2023-02-23 22:58:26,023][D]TC] Server time has been updated! [ST (ms): 1677182307137]
[2023-02-23 22:58:26,038][D]TC] Time difference has been updated! [TD (ms): 1114]
[2023-02-23 22:58:26,337][D]TC] Account order asset balance has been updated! [B: 64.94 BUSD]
[2023-02-23 22:58:26,353][D]TC] Buy order size has been updated! [BOS: 64 BUSD]
[2023-02-23 22:58:26,385][I]TC] Scanning for potential entry opportunities!
vvvvv
  • 25,404
  • 19
  • 49
  • 81
Serhat
  • 11
  • 5
  • Where did you save your `TBLog.sublime-syntax` file? – MattDMo Feb 23 '23 at 22:56
  • Also, are you aware that the `colorama` colors you're using in your log won't show up in Sublime? They are escape sequences that only have meaning when viewed in the command prompt or terminal. They will also interfere with the syntax highlighting you're trying to achieve with the `.sublime-syntax` file. – MattDMo Feb 23 '23 at 22:59

2 Answers2

1

If you open the Sublime Text console (View menu -> Show Console), you will see an error when you save your sublime-syntax file:

error parsing lexer: Packages/User/tblog.sublime-syntax: patterns must be a vector at line 67 column 5

Line 67 is the line under your meta_scope context.

Removing that context, as it isn't referenced anywhere, gives us:

error parsing lexer: Packages/User/tblog.sublime-syntax: capture scope must be a string at line 10 column 7

If you look at the example captures in the documentation, you'll see how you shouldn't string quote the capture group numbers and it expects a plain string instead of a map with a name:.

After fixing all that, it still won't highlight the way you might expect because of the .* you have at the end of your first match pattern in the main context. Removing that gives:

%YAML 1.2
---
# See http://www.sublimetext.com/docs/syntax.html
name: TBLog
file_extensions:
  - tblog
scope: source.tblog
contexts:
  main:
    - match: '^\[.*\]'
      captures:
        0: string.timestamp.tblog
        1: string.loglevel.tblog
      push: log_message

  log_message:
    - match: '$'
      pop: true
    - include: string.quoted.double
    - include: string.quoted.single
    - include: keyword.control
    - include: constant.numeric
    - include: string.timestamp

  string.timestamp:
    - match: '\[\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}\]'
      scope: string.timestamp.tblog

  string.quoted.double:
    - match: '"'
      scope: punctuation.definition.string.begin.tblog
      push:
        - meta_scope: string.quoted.double.tblog
        - match: '\\.'
          scope: constant.character.escape.tblog
        - match: '"'
          scope: punctuation.definition.string.end.tblog
          pop: true

  string.quoted.single:
    - match: "'"
      scope: punctuation.definition.string.begin.tblog
      push:
        - meta_scope: string.quoted.single.tblog
        - match: '\\.'
          scope: constant.character.escape.tblog
        - match: "'"
          scope: punctuation.definition.string.end.tblog
          pop: true

  keyword.control:
    - match: '\b(if|else|for|while)\b'
      scope: keyword.control.tblog

  constant.numeric:
    - match: '\b(-)?[0-9.]+\b'
      scope: constant.numeric.tblog

  string.loglevel:
    - match: '\[(WARNING|ERROR|CRITICAL|DEBUG|INFO)\]'
      captures:
        '0': string.loglevel.tblog
      scope: meta.loglevel.tblog
      push: log_message

and then your text looks like this in ST with your TBLog syntax enabled: example TBLog highlighting

Keith Hall
  • 15,362
  • 3
  • 53
  • 71
0

I used below YAML syntax code.

%YAML 1.2
---
# See http://www.sublimetext.com/docs/syntax.html
name: TBLog
file_extensions:
  - tblog
scope: tb
contexts:
  log_line:
      # data
    - match: '\[([^\[\]]+)\]'
      scope: tb.data
      push: log_line
              
    - match: '$'
      pop: true

  main:
    # debug
    - match: '\[(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2})\]\[D\]'
      scope: tb.debug
      push: log_line

    # info
    - match: '\[(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2})\]\[I\]'
      scope: tb.info
      push: log_line

    # warning
    - match: '\[(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2})\]\[W\]'
      scope: tb.warning
      push: log_line

    # error
    - match: '\[(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2})\]\[E\]'
      scope: tb.error
      push: log_line

    # critical
    - match: '\[(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2})\]\[C\]'
      scope: tb.critical
      push: log_line

    # datetime
    - match: '\[(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2})\]'
      scope: tb.datetime
      push:
        - match: '\[D\]'
          scope: tb.debug
          pop: true
        - match: '\[I\]'
          scope: tb.info
          pop: true
        - match: '\[W\]'
          scope: tb.warning
          pop: true
        - match: '\[E\]'
          scope: tb.error
          pop: true
        - match: '\[C\]'
          scope: tb.critical
          pop: true

And below tmTheme.xml for colors.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>name</key>
    <string>TBLog</string>
    <key>settings</key>
    <array>

      <dict>
        <key>settings</key>
        <dict>
          <key>background</key>
          <string>#272822</string>
          <key>caret</key>
          <string>#F8F8F0</string>   
          <key>foreground</key>
          <string>#F8F8F2</string>
          <key>invisibles</key>
          <string>#5E5E5E</string>
          <key>line_highlight</key>
          <string>#3E3D32</string>
          <key>selection</key>
          <string>#49483E</string>
          <key>findHighlightForeground</key>
          <string>#000000</string>
          <key>selectionBorder</key>
          <string>#222218</string>
          <key>activeGuide</key>
          <string>#9D550FB0</string>
          <key>bracketsForeground</key>
          <string>#F8F8F2A5</string>
          <key>bracketsOptions</key>
          <string>underline</string>
          <key>bracketsContentsForeground</key>
          <string>#F8F8F2A5</string>
          <key>bracketsContentsOptions</key>
          <string>underline</string>
          <key>tagsOptions</key>
          <string>stippled_underline</string>
        </dict>
      </dict>

      <dict>
        <key>name</key>    
        <string>Datetime</string>
        <key>scope</key>    
        <string>tb.datetime</string>     
        <key>settings</key>    
        <dict>
          <key>foreground</key>
          <string>#00B764</string>
        </dict>
      </dict>

      <dict>
        <key>name</key>    
        <string>Data</string>
        <key>scope</key>    
        <string>tb.data</string>     
        <key>settings</key>    
        <dict>
          <key>foreground</key>
          <string>#FFA500</string>
        </dict>
      </dict>

      <dict>
        <key>name</key>    
        <string>Debug</string>
        <key>scope</key>    
        <string>tb.debug</string>     
        <key>settings</key>    
        <dict>
          <key>foreground</key>
          <string>#16ACBA</string>
        </dict>
      </dict>

      <dict>
        <key>name</key>    
        <string>Info</string>
        <key>scope</key>    
        <string>tb.info</string>     
        <key>settings</key>    
        <dict>
          <key>foreground</key>
          <string>#00B764</string>
        </dict>
      </dict>

      <dict>
        <key>name</key>    
        <string>Warning</string>
        <key>scope</key>    
        <string>tb.warning</string>     
        <key>settings</key>    
        <dict>
          <key>foreground</key>
          <string>#EDD436</string>
        </dict>
      </dict>

      <dict>
        <key>name</key>    
        <string>Error</string>
        <key>scope</key>    
        <string>tb.error</string>     
        <key>settings</key>    
        <dict>
          <key>background</key>
          <string>#FF0000</string>
          <key>foreground</key>
          <string>#FFFFFF</string>
        </dict>
      </dict>

      <dict>
        <key>name</key>    
        <string>Critical</string>
        <key>scope</key>    
        <string>tb.critical</string>     
        <key>settings</key>    
        <dict>
          <key>background</key>
          <string>#FF0000</string>
          <key>foreground</key>
          <string>#FFFFFF</string>
        </dict>
      </dict>

    </array>
</dict>
</plist>

And below is how its look like.

https://i.stack.imgur.com/T8iL7.jpg

Serhat
  • 11
  • 5