Google Boosts Python By Turning It Into Go (infoworld.com) 129
An anonymous reader quotes InfoWorld:
Grumpy, an experimental project from Google, transpiles Python code into Go, allowing Python programs to be compiled and run as static binaries using the Go toolchain... In a blog post announcing the open source release, Google stated the project stemmed from its efforts to speed up the Python-powered front end for YouTube. But Google hit an obstacle that's familiar to folks who've deployed Python in production: It's hard to get CPython -- the default Python interpreter written in C -- to scale efficiently. "We think Grumpy has the potential to scale more gracefully than CPython for many real world workloads," writes Google...
Because it doesn't support C extensions, Grumpy doesn't have CPython's Global Interpreter Lock, which is commonly cited as a roadblock to running Python concurrent workloads smoothly. Grumpy also uses Go's garbage collection mechanisms to manage memory under the hood, instead of CPython's. Grumpy creates close interoperation between Python and Go by allowing Go packages to be imported and used with the same syntax as Go modules.
Because it doesn't support C extensions, Grumpy doesn't have CPython's Global Interpreter Lock, which is commonly cited as a roadblock to running Python concurrent workloads smoothly. Grumpy also uses Go's garbage collection mechanisms to manage memory under the hood, instead of CPython's. Grumpy creates close interoperation between Python and Go by allowing Go packages to be imported and used with the same syntax as Go modules.
Re: (Score:1, Funny)
Or is job security with a shitty dead language your plan?
hell no, my current job is programing cgi in perl on apache, using cvs for versioning so i'm all set, thanks.
Re: (Score:2)
Re: (Score:1)
When Google abandons 8.8.8.8 we can switch to 80.80.80.80.
Google buys innovative from outside (Score:4, Informative)
Google didn't invent Python. That was developed by Guido R. at another organization. Years later, Guiodo went to work for Google, well after Python's ascendant popularity.
Android was also developed by another company that Google later bought. Same thing with Youtube. Do you see a pattern yet?
or use react..... whoops (Score:2)
oH whoops, cant use a facebook framework/jsx.
hahaha
why cant google use its own technologies, such as Polymer/webcomponents
But youtube interface sucks , thumbnails have no tooltips, dont show time stamps. When a paused video in full screen goes to window mode, it starts play back again. Also they show ads that are longer than the videos sometimes, (39min ad for 4min video)
Where is the SBS mode 3D interface for 3dTVs ? I can play back SBS 3d videos, but the interface web page it self is not in 3d mode, or ev
Re: Except it doesn't work properly (Score:5, Insightful)
Go offers nothing that an established language can't already do. It'll die just because Google supports it...because Google has no issues with throwing away stuff as soon as they lose interest, no matter how many people are invested in it.
Pretty much every established language can do the stuff the new languages do. It doesn't mean they do them well, or that they can't be improved upon or that we should simply live with the flaws in those languages even if they cause untold numbers of errors in production code. A simple example would be C and C++'s never ending supply of bugs related to dangling pointers, memory leaks etc., many of which are caused by inherent default choices in the language.
While I don't program Go myself it clearly demonstrates a number of advantages for writing multi-threaded, high availability, high performance applications than some other languages. It compiles to a native standalone executable, it's portable, it has garbage collection, it doesn't carry the baggage of a runtime around with it. Performance wise it sits somewhere between C and Java. I've found tools written in Go such as Prometheus to be remarkably stable and reliable.
Re: Except it doesn't work properly (Score:5, Insightful)
it has garbage collection, it doesn't carry the baggage of a runtime around with it
Sigh. Of course it has a runtime. Where else would the garbage collection that you just mentioned be implemented? Or GoRoutines. Or reflection.
I think you're confusing not having a runtime with having the Go compiler statically link the runtime into each executable. That has some benefits that you were alluding to (e.g. "no baggage") but it also has drawbacks such as increased executable size, increased memory usage (with a dynamic runtime, different instances all share the same library in memory), decreased cache usage (since if you have two Go executables, they are constantly evicting each others runtimes from cache, even though they are identical and could be shared) and the maintenance issues having to recompile to take advantage of security/bug/performance improvements in the runtime itself.
I have no issue if you claim that in some (your) use cases the advantages of a statically compiled runtime are worth the disadvantages. But that's not the same as claiming that either the runtime doesn't exist or that it's always advantageous.
Re: (Score:2)
Sharing a framework is fine on servers running *n?x, not so fine on desktop or smartphones.
On Windows, the most widespread desktop and laptop computer operating system in the Western industrialized world, there exists no system-wide automated way to install and update frameworks separately from an application. So when you try to install an application whose framework comes separately, it won't run because the framework isn't already there.
And on both macOS and iOS, Apple requires applications sold in the Ap
Re: (Score:1)
Every language with a standard library has a runtime library. Are you a moron? I'll give you one guess what the RT in MSVCRT means.
Re: (Score:2)
I really have to vehemently disagree with you contention of:
A simple example would be C and C++'s never ending supply of bugs related to dangling pointers, memory leaks etc., many of which are caused by inherent default choices in the language.
If I were to judge your age based on your user ID number, I would day you have been around long enough to know the old saying, "I malloc, therefor I free"
C and to a lesser extent, C++ are quite excellent tools, and for that matter most all other tools are written in those two languages by competent software engineers who actually know what they are doing.
Don,t blame the tool, place the incompetent tool user.
Re: Except it doesn't work properly (Score:1)
Of the popular languages used with real purpose perl python go java c# ruby php which ones used language research crap you are talking about.
Language research is for academic post doc useless stuff. Real languages that are developed by smart ppl solving real problem embraced by ppl whom buy the proposition
I don't count c and c++ cause they are pioneers
Re: (Score:2)
Re: (Score:1)
this isn't the special reading group, when you get 3 stars in the special group, the teacher will let you post here.
Re: (Score:3)
Even though a bunch of new languages coming out. Thought I want to make a new programming language. Next thought should be “No.” This is how really bad programming languages get out there in the world. Javascript is a good example. Javascript was designed and implemented by this guy who went to the University of Indiana. Made a scheme interpreter. Working at Netscape. Need some language to script webpages in web browser. I know I'll just do it with Scheme. Curly braces lot faster than parentheses. Notation that C programmers will like and call it Java-something for marketing purposes. In a week, came up with first Javascript implementation. No going back. On every computer and phone. Can't make any changes, so many JS programs in the world. Arguably the most widely used programming language.
A Logo Proposal (Score:2, Funny)
A Python eating its tail and forming a wheel, making it Go easier.
Re: (Score:2, Informative)
I think the ouroboros symbols is used by PyPy (JIT compilation):
https://en.wikipedia.org/wiki/PyPy
It's all just noise to me... (Score:1, Insightful)
What are they talking about? Stop inventing new shit all the fucking time, and just fucking stick to what fucking works and is stable. You never let anything settle. You constantly crave change for the sake of change. Fuck off. Slow the fuck down. My Windows 10 hasn't even updated to Anniversary Edition yet, and there are already multiple major updates since. It's insanity. Stop changing everything. Go back to the UI of Windows 95 and start over from there. Stop making new programming languages. Stop doing
Re: (Score:2)
Easy, gramps. Forgot to take your medicine again?
please fix syntax while at it (Score:2, Funny)
1. parse python into AST.
2. reformat with curly braces.
3. take over the world; there would be no stopping a curly-braces version of python.
Re: (Score:3)
I would settle for a switch statement.
Don't tax my syns, please. (Score:2)
Re Python:
I would settle for the ability to extend the built-in classes, str in particular. My "settle" went like this:
1) Inquired politely about same
2) Python nerds have orgasm telling me why this is terrible. I am, to put it mildly, dubious.
3) I write 100% compatible pre-processor that gives me the syntax I wanted. [github.com]
4) PROFIT. Okay, well, not really, but EXTENDED STRING CLASS METHOD SYNTAX!
Like...
myString = 'foo'
otherString = myString.doHorribleThing('bar')
pri
Re: (Score:2)
Python's switch statement is a dictionary of inner functions.
Just another f***ing kludge to get around (Score:5, Informative)
Python's GIL. Pythonistas keeping pushing for Python everywhere but don't realize that Python does have its limits and is not the language of choice if you need performance on multiple cores. In my experience, when you try to emulate multithreading in Python using some message passing scheme you end up with something that is more complicated, harder to debug and tune, harder to maintain, than the equivalent written in good C++.
Re: (Score:3)
Re:Just another f***ing kludge to get around (Score:5, Informative)
I think a lot of python users are not even aware of this. It's not a secret, and it's not just for lambdas. Semicolon is a perfectly legal statement separator in python. It's just that newline is ALSO a statement separator.
if x < 1: print(x); print(y); print(z)
is the same as
if x < 1:
print(x)
print(y)
print(z)
No SemiColons in Lambdas (Score:1)
Both 2.7 and 3.6 documentation clearly state that you can't have multiple statements in a lambda function definition:
https://docs.python.org/2/refe... [python.org]
https://docs.python.org/3/refe... [python.org]
In fact the function body part of the lambda is not a statement at all, but a single expression that becomes the function's return value:
https://docs.python.org/3/refe... [python.org]
Re: Just another f***ing kludge to get around (Score:2)
There is a clever concept in Python where you can have multiple statements inside a lambda. You can even split it easily over one or more lines. It is called a "function".
Re: Just another f***ing kludge to get around (Score:1)
"... written is good C++"
That's funny, I've never seen good C++ code.
Re: Just another f***ing kludge to get around (Score:5, Insightful)
A craftsman can write good code in any language.
Everyone else writes crap in every language.
Re:Just another f***ing kludge to get around (Score:5, Informative)
I'm using it right now. Embedded system, Python 2.7 (many legacy modules not 3.x compatible) on a multicore processor. I'm working on an existing Python based project that is very similar to something I did previously at another company in C++. Python is certainly easier to bootstrap a project but once you start dealing with high loads and trying to squeeze all the processing power out of a CPU I'm finding Python much harder to use over C++.
Re: (Score:3, Insightful)
Gee, scripting languages being good at quick and dirty, but then failing to deal with high loads. Who would have guessed.
Re: (Score:2)
im sick of ./configure scripts that spit out syntax error on bracket )
Theres nothing wrong with perl, it looks like C, is easy to write, is NOT messy, unless your brain is fucked up.
Iam sick of coders using 45 meg of modules to save 3 lines of code, as they cannot code anything more than 5 lines of complexity, so just use another lib.
Re: (Score:2)
Iam sick of coders using 45 meg of modules to save 3 lines of code, as they cannot code anything more than 5 lines of complexity, so just use another lib.
When I started looking into node.js.... the amounts of wtfs per minute increased exponentially...
Re: (Score:1)
It's really just to transition a few large in house Google services to Go because Python is slow as fuck and that's never going to change. Most new Google projects just use C++, Java, Go or Dart from the start now. Python is mostly being phased out and this is a tool to help with that.
COBOL (Score:2)
Re: (Score:3)
Real men code in Interpreted COBOL running on top of a Javascript framework.
Try JPython (Score:5, Insightful)
It should use the Java JIT compiler to run much faster than any byte code interpreter.
That said, I do not know why anybody would even think of using a programming language without static typing. Not for performance, but rather for sanity when writing and (more importantly) maintaining code. With type inference it costs virtually no extra typing.
Re: (Score:1)
Good point. I was thinking about the all the discussions of Rust. It seems Rust is just "reinventing" Ada. Used to code in Ada, great language for writing quality robust code. Static typing is a good thing.
Re: (Score:3)
I've played aroun
Re:Try JPython (Score:4, Informative)
My compilers professor said that he believed that type systems are the most important form of program verification that we have (See notes [google.com]).
Re: (Score:2)
Type checking is ultimately formal verification (see the Curry-Howard correspondence). Of course, this limits what type systems can specify about a program, and practically, the restrictions placed on languages based on certain more advanced type systems greatly limit their use in general programming.
Jython, not JPython (Score:2)
Nitpicking, buy it was JPython in 1997
It has been Jython since 1999
https://wiki.python.org/jython... [python.org]
> I do not know why anybody would even think of using a programming language without static typing.
- Dynamically typed programming languages are more productive when writing smaller quantities of rapidly evolving code.
- It is mainly a library and an ecosystem issue. Python tends to have all the modules I need, while Haskell, OCaml and Scala often don't... and they often seem to be much easier to pick up an
Re: (Score:2)
Static typing is a special case of contract capabilities/attributes. It's used because it's easier, but form a theoretical standpoint, you don't want to demand that the object be a "customer", you want to demand that it has a mailing address attribute accessed as .mailing_address. Ideally it should not lose it's "Employee-ness" if it gets passed in, in case a later step needs to special case that or take some extra action if the object is_A or better yet, has_A...
Python is an experiment in that with Duck Ty
it's the libraries, stupid! (Score:3)
There have already been several excellent implementations of Python that compile to native code, either via JIT or directly, including IronPython. But Python's performance problems aren't in Python code, since anything compute intensive is done in native libraries. The real problem with all of those implementations is that, like Grumpy, they don't have native, thread-safe libraries, foremost Numpy and Scipy.
Grumpy sounds like an attempt to migrate Python users to Go. That's probably the right idea, because Python seems to be going nowhere fast. Go, however, is probably not the right language to migrate to, despite its moderate adoption in some systems applications, mostly because Go just gives the finger to existing programming practice for no other reason than FYTW.
Re: (Score:2)
Python-the-language is fine. The problem is that the Python-the-CPython-implementation sucks: it's slow, doesn't support threading, and has a lousy native code interface.
Re: (Score:2)
Yeah, I'd have to agree. But kids like to pretend they made something new.
Re: (Score:2)
Then what's a better word for "compile" that specifically connotes the target language being higher level than assembly language?
(Here, I define "assembly language" as a human-readable programming language with a nearly one-to-one correspondence with a CPU's native machine language or a JIT-compiled bytecode.)
Re: (Score:2)
What's a better word for "execute" that specifically connotes the execution taking place on little-endian powerPC architecture?
Lillpowcecute! WHEEEE!! New words are FUN!
But nobody needs it.
Google compiled some code from Python into Go. Anyone in the industry should know what you're talking about. Anyone that tries to act like a know it all and claim that compiling only refers to the trip to binary deserves to be corrected.
ideas (Score:2)
I like that these languages all bring new ideas and are a great learning opportunity; erlang and go have interesting programming models and CSP/actor functionality. But I feel I should be able to enjoy these things in other languages after they have a while to get absorbed, that's not generally the case though, for example I've been reading about using akka in java for a while now and the syntax looks clumsy to my eye compared with erlang and I've totally failed to get my head around scala.
same thing with i
Re: (Score:2)
erlang gives you nothing that couldn't be done in many other languages. Bad to start a project using niche language when other languages have rich set of libraries, large talent pool for hiring, and many personnel in house for maintenance. erlang has none of the above. immature to get infatuated with the "shiny", real world doesn't want it.
Python to Go, C, language machine (Score:2)
Instead of translating to Go, why not directly translate to C, or even to language machine?
Re: (Score:2)
Go's (initially) looks quite Python-like in terms of packages and memory concepts. Also message-passing is the main mechanism of concurrency.
But Python's power (as well as its woes) comes from the C extensions & GIL. So to get rid of the GIL (required for serious Python performance) you've got to have a fast language replacing it that libraries can be written in that cross calls.
The language for this would also do best to resemble Python to begin with. Go seems like a most logical fit here.
SciPy & N
Kind of like Cython? (Score:2)
How is this different from what Cython has been doing for years? There's nothing new about the idea of translating Python to a statically compiled language to improve performance.
Re: (Score:2)
Unfortunately, the GIL does not only inhibit performance scaling with threads, it causes performance to scale *negatively* on multicore systems. Don't believe me? Try to run a threaded code on a single core system, and then on multicore systems (`cpusets` come in handy for this test). That holds for I/O bound threads just as well as for CPU bound ones or mixtures thereof. Singlecore systems are hard to find nowadays, and cpusets are not always available (in terms of permission and kernel support), and i
Re: (Score:2)
Unfortunately, the GIL does not only inhibit performance scaling with threads, it causes performance to scale *negatively* on multicore systems. Don't believe me? Try to run a threaded code on a single core system, and then on multicore systems (`cpusets` come in handy for this test).
A python process is always using one core at a time (not always the same core because it can block and then get scheduled to run on a different core). If you find a way to bind your Python process to a single core, all your Python threads will be IO bound threads and GIL will not hinder performance. People are just bitching about having to think through the manager/workers pipeline, but any design which wants to take advantage of multiple cores should do that anyway. Multiple cores are best thought of as
Re: (Score:2)
As long as your process does jump from core to core, there is no cache invalidation, so GIL will not hinder you (because it will work as if it was running on a single-core machine).
should have been
" As long as your process does not jump from core to core, there is no cache invalidation, so GIL will not hinder you (because it will work as if it was running on a single-core machine)."
But, as always, you cannot edit a Slashdot comment after it's published.
Re: (Score:2)
*Only* using multiprocessing is hard, too, for two reasons: not having threads makes async programming difficult (think GUIs). It makes I/O or event driven applications much harder.
No one is talking about using only multiprocessing. Threads should always be for IO blocking while processes should be for CPU bound contexts. This is done to avoid the expensive context switch when scheduling waiting on IO while going off to do other work. And each process should be bound (as much as possible) to one core.
Re: (Score:2)
PS.: I should add that, in this context, the GO runtime is interesting, as it puts away several of those limitations. So I do understand the rationale behind this I guess. As others above, I don't really see the advantages of simply using GO directly, or any other mature language. My guess is that there is too much Python code around to simply abandon or re-implement it.
Re: (Score:3)
Well, and that's what threads are: something like multiple UNIX processes sharing all their memory via shared memory and a few other niceties. Unfortunately, Python doesn't support that kind of fast, consistent IPC. What it supports is either a limited form of shared memory, or slow IPC that involves copying.
So, you can look at the Python runtime either as lacking thread support or a
Re: (Score:2)
Yes, the OS supports shared memory; it's Python's support that is poor.
That's bullshit. Numerical and AI applications benefit greatly from sharing memory that way, compared to other IPC mechanisms.