TMWL Dec’20 — PyCharm container interpreters, Ethereum in Scala and AWS re:Invent

Programmers share what they’ve learned

Every month we share what we’ve learned in our team. In December Mikołaj, Piotr and Michał discovered:

• how PyCharm is not keeping up with different dependency versions in containerized environments and what to do about it,
• when an empty string can have a monetary value,
• and what’s new in AWS!

Updating PyCharm container interpreters — by Mikołaj Koziarkiewicz

PyCharm is a nice tool, including all sorts of useful features, like the relatively-recently added ML-based code completion. Unfortunately, it can also be a bit…​ uneven at times.

One of such problematic areas is operation with containerized Python interpreters and environments. While PyCharm made great strides over the last year or so, it still has some quirks that need to be worked around, like diagram display, which I covered as a part of a dedicated blogpost.

The following note also deals with working on a containerized interpreter — specifically a docker-compose one — but here, I’m going to discuss another aspect of this kind of tool integration.

Namely, recently I’ve been rather rapidly experimenting with different dependency versions in containerized environments, and it turned out that PyCharm simply wasn’t keeping up — it was still displaying “old” versions.

Now, the first thing I had learned (a while ago) is that, in PyCharm, these issues are often the same for containerized interpreters and remote interpreters, e.g. SSH connectivity; I’m guessing there are some common underlying components there (which makes sense). So, off I went to look for an answer.

Unfortunately, none of the workarounds listed in issues such as this one managed to provide a resolution (including the legendary Invalidate Caches and Restart). Even the new “Rescan Available Python Modules and Packages” feature mentioned in this issue didn’t help at all.

Fortunately, I’ve eventually arrived at two possible solutions: an easy way, and a slightly more involved way. Hopefully one of those can help you with the problem, depending on the PyCharm version you’re using.

The easy way, suggested to me by our very own Maciek Opała, (thanks!) is really easy. All you have to do is go to the directory holding your docker-compose spec, and rebuild it with:

docker-compose build

That’s all! This certainly works on the newer PyCharm versions, like 2020.3.

The more involved way consists of the following steps:

• Ensure you only have the affected project running in PyCharm (i.e. close all others). This is just a safeguard against deleting another interpreter accidentally.
• Go to the Python Interpreter settings for your project.
• Click Show all, and then delete the current interpreter with the “minus” button.
• Remove the docker_interpreter:latest image from your local repository. In the console, this would be:
• docker rmi docker_interpreter:lates although you may need to force it with:docker rmi -f docker_interpreter:latest depending on the state of the current containers.
• Add the interpreter again.

You should now have the updated dependencies proudly displayed in your Python Interpreter settings. No restart, no cache invalidation required. Unfortunately, you have to repeat this every time you change your dependencies — but it beats the alternative.

The value of an empty string — by Piotr Buda

Would you say that an empty string can have any monetary value? Let me tell you, it can. Recently, together with Grzegorz Hejduk, we started a new project that would use Ethereum smart contracts to execute some arbitrary logic, whereas Scala backend using Web3j would monitor Ethereum blockchain and invoke our smart contracts when necessary.

We tested our integration on the Ropsten test network, had tests using Ganache and everything looked good, so we proceeded to deploy smart contracts on the mainnet.

Our backend finally detected an event of interest. It proceeded to execute our main smart contract and that’s when things went sideways. The result? On Etherscan I noticed that there’s a failed transaction that used all gas provided for it. I tried to debug that transaction with truffle and couldn’t load anything meaningful, also no breakpoint I’d set would stop the debugging process. I was puzzled and that’s when I discovered that:

• sending a transaction to an empty address actually… creates a smart contract on the network
• we didn’t provide our smart contract’s address in configuration — it remained a default empty string!

Since our transaction was actually a method invocation on a contract, it failed, but drained all gas in the process and because we used web3j’s DefaultGasProvider and its settings that configuration mishap cost us 0.16 ETH or $100 at the time.

How to avoid such mistakes? We use ZIO config module with Typesafe config but now we want to move that to pureconfig and enforce all Ethereum addresses (defined by tagged types) to be non-empty. But that’s another story for another post.

AWS re:Invent — new features and services — Michał Matłoka

AWS annual conference — re:Invent has been held online this year. This month I’ve learned about new services and features AWS is planning to bring to us in the near future. So far the most interesting for me are:

1. AWS Lambda 1ms billing and Container Image Support — previously code executions times were rounded to and billed per 100 ms. Since now lambdas will be billed per 1 ms. In practice this may result in big cost savings. What is more, AWS Lambda supports container images now. This may be quite useful for cases where non-standard execution environments are leveraged.
2. Amazon Managed Service for Prometheus and Amazon Managed Service for Grafana — Prometheus and Grafana are open-source monitoring-related solutions used in many projects. AWS starts to offer their managed versions.
3. AWS Fault Injection Simulator — have you heard about Netflix Chaos Monkey before? It is a tool related to introducing failures in your system in order to detect application weaknesses. In 2021 AWS plans to launch their own tools, whose role will be quite similar — to inject failures, throttle APIs, simulate outages and others.
4. AWS CloudShell enables access to AWS CLI via AWS Management Console.
5. VPC Reachability Analyzer is a new tool helping to troubleshoot issues related to VPC misconfigurations and services reachability. It is quite simple — you just need to define source and destination of the traffic and the analyzer will check if they are accessible and what is the request path.
6. Amazon S3 — Strong Read-After-Write Consistency — it may not be a widely known information, but S3 was so far eventually consistent. After new changes all reads performed after writes are now strongly consistent.

AWS re:Invent has been a huge event with a lot of talks and tracks. It is not over yet so we can count on new features and product announcements within the next few days.

And what have you learned in December? Let us know! :)

BTW, we are always looking for outstanding professionals to join our team!

Questions? Ask us anything about remote work, how does the cooperation with us look like, what projects do we have, or about anything else - on the dedicated Slack channel 💡