这是indexloc提供的服务,不要输入任何密码
Skip to content

Conversation

@dalf
Copy link
Contributor

@dalf dalf commented Jul 26, 2022

What does this PR do?

Based on #1412

Add Python types about the results.
The purpose is described the fields with code using TypedDict.

Why is this change important?

Add documentation about the different result types.

How to test this PR locally?

Author's checklist

pyright errors have been fixed.

Related issues

Related to #1412

@dalf
Copy link
Contributor Author

dalf commented Jul 28, 2022

In documentation, sphinx.ext.inheritance_diagram could be helpful to draw the inheritance diagram.
But TypedDict does not preserve the original bases ( __mro__, __base__, __bases__ don't contain the inherited class), so the Sphinx extension can't find the class hierarchy.

return42 added a commit to return42/searxng that referenced this pull request Aug 2, 2022
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
@return42
Copy link
Member

return42 commented Aug 2, 2022

@dalf FYI I picked up this PR and continue to develop in (my) branch https://github.com/return42/searxng/commits/refactor_results .. its fare from usable .. I will ping you when I have something I can show you.

@return42
Copy link
Member

return42 commented Aug 3, 2022

But TypedDict does not preserve the original bases

TypedDict is a type in the world of "Type Hints". You can inherit "Type Hints" with similar concepts known from the OO paradigm and the syntax of "Type Hints" definitions use the same keyword class what might mess these both worlds:

  • "Type Hints" are (type) hints to describe types --> objects are never instantiated from a "Type Hint class"
  • In the OO paradigm "Classes" are used to "classify" objects --> objects are always instances of classes.

UPDATE: Since Type-Hints are not classes (can't be instantiated) I prefer common classes that inherit from a dict-Type.

I mean: if we implement type-hints we can only check types (and add some doc-strings) but we can't implement methods like we know from OO paradigm .. TBH: IMO these "typing" stuff is sometimes an overkill. Python is a language with dynamic types and type-hints can help to valid some types in static checks but I also think it should used sparse and with care.

–---

image

@dalf
Copy link
Contributor Author

dalf commented Aug 3, 2022

The idea of this PR is to make a clear list of the result types and theirs fields (models.py) without changing anything about the implementation. Actually it is the documentation you wrote but converted as Python typing.

IMO these "typing" stuff is sometimes an overkill. Python is a language with dynamic types and type-hints can help to valid some types in static checks but I also think it should used sparse and with care.

I'm sure if you talk about TypedDict or typing in general.
If the code editor supports them (typed annotation), IMO:

  • it is easier to read than the documentation.
  • it is super helpful: right away without running the code, I know when there is an error (typo, wrong argument type, etc...). Even with TypedDict.

But talking about OO classes and instances: https://gist.github.com/dalf/972eb05e7a9bee161487132a7de244d2 . There is

  • an URL class to have one value for the URL and the parsed URL. The result classes convert the string to an URL instance automatically.
  • dataclass for the results (EngineResult, AnswerResult, DefaultResult, ImageResult, InfoboxResult)
  • EngineResultContainer is a generic container for the result. There is only one subclasses (DefaultResultContainer), but it should be one per result type.
  • the response function in the ####### EXAMPLE ####### section gives an example of what the response function of an engine could be.

@return42
Copy link
Member

return42 commented Aug 3, 2022

I'm sure if you talk about TypedDict or typing in general.
If the code editor supports them (typed annotation), IMO:

Yes, I was talking about TypedDict 👎 .. to me, type-hints in function's prototype are very useful 👍 👍

TBH I don't really know what the use-case for e.g. TypeDict is .. when we use @dataclass and define fields there is no longer a need for the TypedDict model of be52ff8 .. right?

But talking about OO classes and instances: .. There is ..

  • an URL class to have one value for the URL and the parsed URL.

I don't have a final meaning about / I will ignore the URL class for now (we can have a look at this later), I like to finish the class model of the result types first ..

  • EngineResultContainer is a generic container for the result.

LGTM but I will also ignore for now .. like URL this is a bit OT from my POV (class model of result)

the response function in the ####### EXAMPLE ####### section gives an example of what the response function of an engine could be.

I don't prefer this approach; for me an engine is something like a plugin that returns a simple data type (a python dict or list). I don't want to bind the return value of the engine to a special data-type that is used (internally) in SearXNG. I prefer to stay with the current concept, where SearXNG determines the result type by the existence of a property.

The coupling of engine and server code should be as simple as possible ... then it might be possible to add a layer at this point in the architecture to send a request to a remote SearXNG node for example (just dreaming a bit).

I really like the @dataclass approach from your gist .. I will try to take it over in my branch.


Remarks about the gist:

In the gist you implemented a method EngineResult.merge .. I have a doubt if the merge is a task of the EngineResult ... I think (from an architectural POV) the ResultContainer should decide how results are merged / not the Engine-Result (hope I was clear). I know that operations like __add__ are often implemented in the type .. but IDK if this merge is really a task of the result-type .. may be I have the wrong view on this.

@dalf dalf marked this pull request as ready for review August 13, 2022 09:21
@dalf dalf force-pushed the refactor_results branch 2 times, most recently from 051ba92 to 5667064 Compare January 28, 2023 15:12
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants