Wildcards for intermmediate paths

[hbarcelos]

I was making some tests here and figured out that wildcards * only work in tail position.

Currently we have to do something like this:

[
    'err.options.auth.password',
    'options.auth.password',
    'auth.password'
]

It would be very nice if we could do something as:

*.password: objects containing password key in the first level.
err.*.password: objects containing err key in the first level and password key on the third.
**.password: objects containing password key in any level (possible performance hit)

[mcollina]

Of course! Would you like to send a PR?

[hbarcelos]

I'm having a busy week, but maybe I could pull something out during the weekend.
Would be a problem if this feature means adding a new dependency for path parsing?

[davidmarkclements]

@hbarcelos not at all.

The most important thing here, is that it's high performance.

1. Any changes must avoid adding any overhead to current use cases
2. The more complex wildcard support should be as fast as possible

So along with units tests we would want comparison benchmarks (master vs your branch – on your own machine is fine)

If there's any benchmarks missing for a particular case (e.g. the speed of wildcard matching), that would need to be added so we could have comparison benchmarks of before/after the changes

[tomwidmer]

This would be very useful - currently it's too easy to accidentally log an error with the request object embedded somewhere within it that includes an authorization header. If I could just exclude any key called 'authorization' at any level, that would reduce the risk of accidentally logging credentials massively.

An efficient implementation won't be trivial for the general case. Handling the special case of '*.x' would be easy to implement of course, and I might have time for at some point.

[danielo515]

I also need this for my production environment. In my case the penalty hit is on not redacting the partially nested array of 10k elements.
Ideally, if you could redact elements based on length would be awesome too...

[mcollina]

@danielo515 have you tried pino@5? It has significant improvements over pino-noir.

[danielo515]

Hello @mcollina , thank you very much for your prompt response
No I did not.
There are places where I can not use it (we have some legacy stuff on hapi16 that in turn requires older pino). But the places where performance is critical I think I can use pino 5 if we are not using it already.

What are those improvements ? Where can I check the docs ? And how does pino-noir relates to pino ?

[danielo515]

Here am I again.
If you are talking about https://github.com/pinojs/pino/blob/master/docs/redaction.md that is impressive.
Basically it invalidates pino-noir at all.
However to me is still not clear how the wildcard works.

If I do 1[*].results does that mean that I will redact all the entries of the possibly array that contains objects that has a results field ? Or does that mean that it will deeply scan the objects and censor everything that has a results property ?

NOTE: I'm using a 1 as a first property because it is an array of arrays...

[mcollina]

I don’t think there is any special logic about arrays, and the best thing to do is to try and check.

I would not touch this code (it is unreadable and very very tricky to handle), but rather open an issue and maybe send a PR to

[mcollina]

https://github.com/davidmarkclements/fast-redact

[davidmarkclements]

Yes wild cards for arrays or objects are addressed the same - so if 1 is an array 1[*].results will redact the results key in every object within the array (but only the top level, not nested objects). If 1 is an object, the results key in every object contained in the 1 object will be redacted

[danielo515]

@davidmarkclements are you talking about pino-noir or pino 5?

[peterpod-c1]

👍

[carlos-xpln]

At the end, was this feature considered? Would be awesome to have it. If not, I would be open to send a PR :D