Hey there!
I’m a chemical physicist who has been using python (as well as matlab and R) for a lot of different tasks over the last ~10 years, mostly for data analysis but also to automate certain tasks. I am almost completely self-taught, and though I have gotten help and tips from professors throughout the completion of my degrees, I have never really been educated in best practices when it comes to coding.
I have some friends who work as developers but have a similar academic background as I do, and through them I have become painfully aware of how bad my code is. When I write code, it simply needs to do the thing, conventions be damned. I do try to read up on the “right” way to do things, but the holes in my knowledge become pretty apparent pretty quickly.
For example, I have never written a class and I wouldn’t know why or where to start (something to do with the init method, right?). I mostly just write functions and scripts that perform the tasks that I need, plus some work with jupyter notebooks from time to time. I only recently got started with git and uploading my projects to github, just as a way to try to teach myself the workflow.
So, I would like to learn to be better. Can anyone recommend good resources for learning programming, but perhaps that are aimed at people who already know a language? It’d be nice to find a guide that assumes you already know more than a beginner. Any help would be appreciated.
It’s always good to learn new stuff but in terms of productivity: Don’t attempt to be a programmer. Rather attempt to write better research code (clean up code, revision control, better commenting, maybe testing…)
Rather try to improve cooperation with programmers, if necessary. Close cooperation, asking stupid questions instead of making assumptions etc. makes the process easy for both of you.
Also don’t be afraid to consult different programmers since beyond a certain level, experience and expertise in programming is vastly fragmented.
Experienced programmers mostly suck on your field and vice versa and that’s a good thing.
Odd take imo. OP is a programmer, albeit perhaps not a very good one. Did a PhD (computational astrophysics), been working as a professional dev for 10 years after that. Imo a good programmer writes code that solves the problem at hand, I don’t see that much of a difference between the problem being scientific or a backend service. It doesn’t mean “write lots of boilerplate-y factories, interfaces and other layers” to me, neither in research nor outside of it.
That being said, there is so much time lost in research institutes because of shoddy programming by researchers, or simply ignorance, not knowing a debugger exists for instance. OP wanting to level up their game would almost certainly result in getting to research results faster, + they may be able to help their peers become better as well.
25 years in the industry here. As I said there’s nothing against learning something new but I doubt it’s as easy as “leveling up”.
Both fields profit a lot from experience and it’s as much gain for a scientist do become a software dev as an architect becoming a carpenter. It’s simply not productive.
there is so much time lost in research institutes because of shoddy programming
Well, that’s the way it is. Scientific code and production code have different requirements. To me that sounds like “that machine prototype is inefficient - just skip the prototype next time and build the real thing right away.”
To me that sounds like “that machine prototype is inefficient - just skip the prototype next time and build the real thing right away.”
I don’t think you understand my point, which is that developing the prototype takes e.g. 50% more time than it should because of complete lack of understanding of software development.
As a researcher: all the professional software engineers here have no idea about the requirements for code in a research setting.
I recommend you use
- git. It’s nice to be able to revert changes without worry.
- descriptive variable names. The meaning of descriptive is highly dependent on your situation. Single letters can have an obvious meaning, but err on the side of longer names if you’re unsure. The goal is to be able to look at a variable and instantly know what it represents.
- virtual environments and requirements.txt. when you have your code working you should have pip (or anaconda or whatever) take a snapshot of your current python installation. Then you can install the exact same requirements when you want to revive your code a few months or years down the line. I didn’t do that and it’s kinda biting me in the ass right now.
As a researcher: all the professional software engineers here have no idea about the requirements for code in a research setting.
As someone with extensive experience in both: my first requirement would be readability. Single python file? Fine with that. 1k+ lines single python file without functions or other means of structuring the code: please no.
The nice thing about python is that your IDE let’s you jump into the code of the libraries you’re using, I find that to be a good way to look at how experienced python devs write code.
You can jump to definition in any language. In fact, python may be one of the worst ones, because compiled libraries are so common. “Real signature unknown” is all you will get some times. E.g. Numpy is implemented in C not python.
My point about the jumping into was that you can immediately start reading the sources. Most alternative languages are compiled in some form or other so all you’ll see is an API, not the implementation.
My comment was not asking for clarification, I am contradicting your claim.
Granted, my experience is mostly limited to python and rust. But I find that in python you reach the end of “jump to definition” much much sooner. Fundamental core libraries of Python are written in C, simply because the performance required cannot be reached with python alone. So after jumping two levels you are through the thin wrapper type and your compiler will give you an “I don’t know, it’s byte code”.
In Rust I have yet to encounter this. Byte code is rarely used as a dependency, because compiling whatever is needed is no issue - you’re compiling anyway - and actually can allow a few more optimizations to be performed.Edit: since wasm is not yet wide spread, JavaScript may be the best language to dig deep into libraries.
Mostly ML or data processing libraries I would assume, I’ve read tons of REST server and ORM python code for instance, none of that is written in C.
Wrt rust: no experience with that. I do do a lot of C++, there you quickly reach the end as typically you’re consuming quite a bit of libraries but the complete sources of those aren’t part of what is parsed by the IDE as keeping all that in memory would be unworkable.
REST server and ORM python code
Fair enough, that can be achieved with pure python.
The thing to think about is reusability. Are you copying and pasting code into multiple places? That’s a great candidate to become a class. If you have long lived projects (i.e. something you will use multiple times over a lot of years) maintainability is important. Huge functions and monolithic applications are very hard to maintain over time.
Break your functionality out into small chunks (methods and classes). Keep it simple. It may take a while to get used to this, but your time for adding additional functionality will be greatly improved in the long run.
A lot of great programmers were terrible at one time. Don’t let your current lack of knowledge of principles stop you from learning. One of the biggest breakthroughs I had as a programmer is changing how I looked at architecting applications. Following SOLID principles will assist a lot in that. Don’t try to understand and use these principles all at once, take your time. Programming isn’t what you make your living with, it’s a tool to help you be more efficient in your current role.
Realize that becoming a more effective programmer is different for everyone. Like you, I was self taught. I was a systems and network engineer that decided to move into software development. I’ve since moved into a role that takes advantage of all the skills I’ve learned through the years in SRE. like you, a lot of what I write now is about automation and analysis.
Careful with this. Not everything needs to be reusable, and copy/paste isn’t inherently bad.
You aren’t wrong… But everything with extended use needs to be maintainable. Making a change in 5 places sucks.
Plus, that’s what open-closed principle is all about. Instead of adding additional functionality to current working code, you extend and modify.
Making a change in 5 places sucks, making it in 2 could be reasonable. If 2 pieces of code are similar but different enough, I’ve seen way too often people try to force them into a common abstraction. That’s more what the article is about.
I’d say go with Go or Rust. Go is like Python (garbage collection) but compiled. Rust is kind of like C++ but not exactly. It does not have garbage collection or manual memory management but something called “ownership and borrowing”. It’s as fast as C++ or even faster and has a modern syntax. Though Rust is harder than Go since it is under the hood a systems programming language. If you want something faster than Python, Go is good. I specifically chose Rust over Go since I wanted performance and just wanted to try how it was. I’m still a beginner in Rust but I wrote a few projects at reasonable scale for my level. And also, Rust’s error messages are extremely nice. It really lives up to the memes.
To learn Rust: https://www.rust-lang.org/learn
To learn Go: https://go.dev/learn/
Rust syntax is way closer to Python than go. Go’s syntax is awful imo. It’s like objective C
Use an IDE if you aren’t already. Jetbrains stuff is great. Having autocomplete is invaluable.
