flomaster

sandbox

why

The current process, used by every (top 3 at least) syntax highlighters:

1. Lexical tokenization of raw source code
2. Wrapping each token in a DOM element
3. CSS selection

Step 2 is an awkward mechanism which triggers layout reflow¹ in
the main thread, hence contributing to “jank” and First-Meaningful Paint
delays. Additionally, it enlarges the DOM further exacerbating the issue.

This prototype attempts an alternative method using position offsets,
so it (should) avoid these issues altogether.
Simply put: it’s faster.

todos

• get a non-fancy tokenizer to iterate upon:
  • return shape: [{ token: 'keyword', start: 5, end: 10 }]
  • add minimal tests.
  • the tokenizer falls over some regex,
    see notes in test.
  • fix/ensure errors are correctly printable in browser
  • many cases that need ironing-out. Fix the presets.
  • needs precedence fixes, i.e forEach is colored half for for and the rest separate.
• start benchmarks asap, this is more than enough
• This is perfect: https://gist.github.com/addyosmani/c053f68aead473d7585b45c9e8dce31e
• what about other browsers
• fix the online sandbox
• a decent color mapper
• add a small benchmark
  • decide on the parameters before you start
• review if it makes sense

test

tests require Node v22+

node --run test

misc

serve demo sandbox locally

$ node --run demo

license

MIT License

Copyright (c) 2024 Nicholas Kyriakides

Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and documentation files (the “Software”), to
deal in the Software without restriction, including without limitation
the rights to use, copy, modify, merge, publish, distribute, sublicense,
and/or sell copies of the Software, and to permit persons to whom the
Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.

1. this is distinct from forced synchronous layout reflow, the reflow that is triggered by accessing certain geometric APIs. Both are types of reflow. ↩