So last week we switched from chompjs to pure python code in d07c006. I do think it's an improvement to get rid of a dependency, especially when it is only used by a single engine.

However, it already broke, so brave.news and brave.videos weren't working, which I fixed in #2784. After having a look at the function js_variable_to_python, I noticed it is also 5x slower than chompjs, taking around 10ms on a normal brave page, which is not bad but can definitely add up and become noticeable, especially on weaker hardware. I'm also concerned that it can come with a bigger maintenance cost than using chompjs.

Here are some other solutions:

chompjs

Pros:

• 400% faster than js_variable_to_python
• Less likely to break in the future
• Much less code on our end

Cons:

• External dependency written in C
• We would currently only use it for one engine

Function from yt-dlp

Pros:

• More robust than our version
• Pure python
• Unlicensed, meaning we can use it no strings attached

Cons:

• Twice as slow as js_variable_to_python
• A lot of code in our project

My naive implementation

I rewrote js_variable_to_python to be even more bare-bones, just as an example. This works exactly the same for parsing brave responses.

import json
import re

QUOTE_RE = re.compile(r"([\{,])(\w+)")


def parse_js_object_naive(obj):
    parts = obj.split('"')
    for i, p in enumerate(parts[::2]):
        parts[i * 2] = QUOTE_RE.sub(r'\1"\2"', p.replace(":.", ":0.").replace("void 0", "null"))
    return json.loads('"'.join(parts))

Pros:

• 50% faster than js_variable_to_python
• 6 lines of code instead of 70
• Potentially less maintenance cost
• Potentially easier to understand

Cons:

• Makes more assumptions and is just as likely to break as js_variable_to_python
• Pretty hacky

Testing

In [1]: import chompjs

In [2]: %timeit -r 15 js_variable_to_python(data)
9.31 ms ± 7.75 µs per loop (mean ± std. dev. of 15 runs, 100 loops each)

In [3]: %timeit -r 15 parse_js_object_naive(data)
6.15 ms ± 53.6 µs per loop (mean ± std. dev. of 15 runs, 100 loops each)

In [4]: %timeit -r 15 chompjs.parse_js_object(data)
1.82 ms ± 2.25 µs per loop (mean ± std. dev. of 15 runs, 100 loops each)

In [5]: js_variable_to_python(data) == parse_js_object_naive(data)
Out[5]: True