Comprehensions are currently compiled as nested functions, which provides isolation of the comprehension’s iteration variable, but is inefficient at runtime. This PEP proposes to inline list, dictionary, and set comprehensions into the code where they are defined, and provide the expected isolation by pushing/popping clashing locals on the stack.
Last: F Strings will support some common use cases that broke interpolation in the past, like f'{ myDict['myKey'] }' and f"{'n'.join(a)}"
My fear is that this type hinting pushes #python down the same bad road as #cpp:
Making an originally nice programming language more and more complex and ugly by giving in to the pressure of adding more and more features which don't fit into the original idea of the programming language.
In my view, the core philosophy of #python is that it is a high-level dynamically typed language . And the core philosophy of #cpp is that it is a low-level statically typed language.
Yes, I think the type hinting at all was a bad idea.😉
I heared some arguments and experiences about the usefulness, but I don't see it yet.
The argument "it is optional" is the typical reason for giving up a sweat nice design and creating big messy bloated system instead.😉 A feature should not only be optional, but really useful and necessary to offset the inherent disadvantage of every new feature of making a system more complex.🙂
@folkerschamel@kfet@ado You will definitely see the usefulness of type annotations when you ever have to work on a large codebase project without them.
But as serious comment, I would guess that mypy would get panic attacks when running on our code, because so many architectures are inherently based on dynamic typing.😁
@folkerschamel@kfet@ado As a software engineer, my job is to create things. The more correct I can create them, the more they last, the less they cost long-term and the happier everybody is. Dynamically typed languages are inherently not good at creating correct software, by very definition.
You can see that the whole software industry is realizing this slowly. The way TypeScript is catching up to JavaScript in popularity is a perfect example.
@folkerschamel@ado I'll have to disagree with you there. Looking at OP's post it's the opposite - the new type hinting both hugely simplifies things, while makes it much more readable.
But you don't have to use type hints at all, even in the latest versions. They're completely optional. You could write an entire new application without a single type hint in it, and it would work just fine. Static linters like mypy wouldn't be able to find as many problems without the hints, but that's a tradeoff.
Or does it somehow bother you when some #dependency has type hints in its code?
I get it; I've been a big fan of Python since 1995. When type hints were first proposed it did seem a little at odds with Python philosophy - but I was reassured by the immediate assurances that they were optional, and were going to stay optional. So I mostly ignored them for a few years.
When I did start using them, it felt a little weird; the code didn't "look" right. But that's the same with any change.
Now I'm a big believer. They have prevented so many bugs...
Do you have representative specific samples of bugs you found that way? Trivial bugs or nasty bugs, dangerous skeletons in the closet? Could automated tests have caught them, too?
Did you consider switching from #python to statically typed languages like #java, #rust or #golang ?
There was a time when I couldn't imagine in the world to use a dynamically typed language for serious programming. I changed my view completely. B
May be the same with type hints. But maybe not.😉
@folkerschamel@cazabon@ado IMO, the even if it didn't find bugs (which it does), it helps build confidence in the changes you're making. And any junior dev can throw code against the wall; much of what we do as we climb the ladder is building confidence in the code (or rather helping the junior devs build confidence in the code). (see also: Pirsig's ideas about quality)
Asking for specific #examples type checking finds is ... unproductive. You write 2 modules of code, run #mypy, and get a list of the 11 places you need to fix - before you've even written unit tests that may or may not have caught the same problem.
If I tried to keep track of them all, I wouldn't have time to write code.
And yes, many of them are #bugs that would be #agonizing to find by other routes. Your unit test #probably doesn't try passing a #float to that #function you wrote that you only considered using ints with - but with hints, mypy will find that "one weird way" that a float value returned from somewhere else could get passed into the function.
Type hints in code - used well and consistently, no using "#Any" to shut mypy up - make me 50% more productive, I think. </anecdote>
I remember recently learning about one case where some code intentionally mixing integer and float calculations was writing out the wrong type into Elastic Search, causing an inconvenience in Kibana visualizations because integers don't allow the same aggregation functions as floats.
But this is the only case I can remember static type checks probably would have caught, and it was quite harmess.
Maybe I miss something, or maybe our code base is just different, who knows.😉
From the more theoretical perspective - knowing that this is definitely not an exhaustive perspective - right now I cannot imagine situations where a type bug wouldn't be also a functional bug, which would be caught anyway by code review or testing.
Mixing floats and integers comes closest. But also here I think #python does a good job avoiding mistakes. For example, the type of a result of all operations does depend only on the input types, but never on the input values.
I just realize: As far as I understand #mypy, my previous #python Elastic Search example wouldn't be caught by #mypy because the parameter was passed to a logging function using formating via an f-string.
@folkerschamel@cazabon@ado I think the main value of static type checking is finding these bugs faster and with less effort. I agree that it's not common to have a type mismatch that wouldn't show up as a functional bug under the right circumstances, but code review does not necessarily catch all those bugs, and neither does testing. Or at least, sometimes you have to write a pretty weird test (or expose the program to real live users gasp) to find them. I'm envisioning something like a variable which some rarely-executed part of the code sets to None and a different part of the code assumes is always non-None, or passing the wrong number of arguments to a last-resort error-handling function, or stuff like that.
The other main value of type hints IMO is documentation, so that users of a library/function/etc. know what type of thing they're supposed to pass in. I find it a lot easier to understand (and write!) that info in the form of type hints rather than prose.
I see your first point theoretically. But I don't remember situations where I thought "static type checking would have helped us avoiding that trouble". But I keep it in mind.🙂
About documentation, I see the argument in theory. But I think in practice you get that information for free by naming (e.g. "counter" -> int, "name" -> str) or documentation ("number of elements" -> int, "object supporting xyz").
In general, specific real-world examples would be interesting.🙂
@cazabon@folkerschamel@ado a common case i find is failing to take the None/Optional possibility into account, your function can be completely covered, but fail to handle the case where a parameter is None, but it's something that actually happens in your codebase. mypy will tell you about it, either in your function because you declared the parameter as Optional, but do things that are illegal with it, or outside, because you pass a potentially None value to a function that doesn't expect it.
Interestingly such cases are not caught by compilers of statically typed languages like #cpp or #java (assuming you are talking about situations where you would use pointers/references in #cpp/#java).
For our software we have the explicit coding style that functions must support None/null/null-pointer parameters for class instances (or more general, never expect conditions met). So maybe this convention helps us creating robust code for such cases.
@folkerschamel@tshirtman@cazabon@ado If you pass a nullptr into a function that operates on an object, then - simply put - you have an error in your code. Use pointers if an object is optional (and handle that of course, as the author of that function you should know), otherwise use references.
We have the opposite coding style:😉 a) Parameters and return values must not be references, but pointers, b) function must support null pointer parameters, and c) callers must support null pointer return values.
In our interfaces there are many situations where passing or returning "no object" is quite useful. So to keep it simple and reduce null pointer exception risks, we make it a rule to always support it.
Interestingly it's related to fault-tolerant code.
For example, a function find_some_object (loading a resource) may not find the object, report an error, return null.
The caller should continue, possibly passing that null pointer to other functions without raising an exception or even aborting.
@folkerschamel@DanielaKEngert@cazabon@ado having such an explicit convention is indeed a strategy to avoid this class of problems, but do you have away to check/enforce the convention, or do you have to rely on humans to check it? I assume test coverage requirements might make it more obvious when it's missing, but i have a bias with giving to humans tasks that can be automated, unless the cost is prohibitive.
I also fear this way of handling it might lead to duplicate work in calling/callee
I fullly agree that you should avoid relying on human behavior. Automation is much more powerful. Also, frankly, our automated testing is not as comprehensive as it should be (historical and practical reasons). But in practice at least I don't remember cases where static type checking in #python would have saved us trouble.
Duplication: Yes, our code is full of null pointer/None checks. You see tons of if(obj) / if obj: statements.😉
@folkerschamel@DanielaKEngert@cazabon@ado i don't really do Java by the way, but my coworkers that do are using type definitions that make Nullable/NotNullable explicit, which i assume is checked at compile time, i hope similar things exist in cpp, but honestly, i'm not keen on learning more about that language 😆.
References are (at the language level!) aliases of objects. They contribute a new name for an object, possibly in a different scope. Therefore it is impossible to refer to <not an object>
Pointers are objects on their own. They may or may not refer to another object. In Haskell lingo: pointers are maybe monads
In some (many) cases, compilers lower them to processors in the same way - if they have to.
@cazabon@folkerschamel@ado here's the thing: 9/10 of the projects I deal with at work I didn't get to choose what the code base looks like when I get my hands on it.
So if someone has used type hints, then that's what I have to work with, for better or worse.
If you add a feature to a language it's never going to be optional to maintenance developers.
Add comment