Céondo - Fluid Phase Equilibria, Chemical Properties & Databases

The Visitors of Cheméo Changed

2026-06-25T20:10:00+02:00

Last week I pulled just under six days of access logs off the Cheméo server. I only wanted to check if my handling of the bot spamming the subscription form was efficient. I came away with something I did not expect: the audience of the site has almost entirely changed.

The boring part first

Cheméo runs on a single box. Caddy terminates TLS and serves the static files. Behind it are two Go binaries: a boring web server that answers the requests, and a custom indexer that runs quietly in the background, coupled with a Python compute node running the property prediction code. SQLite is the store underneath. No Kubernetes, no autoscaling, nobody on call. One server, in one rack.

Over a full month that box serves about 10.6 million requests and nearly 1 TB of data. That is roughly 5.6 million HTML pages and a steady 3 to 4 Mbps, day and night. Uptime is close to 100%, and I have not had to log in and fix anything in a long time.

And the box is doing real work the whole time. The busiest path on the whole site is /similar, the chemical similarity search, with about 150,000 hits in the window. Every one of those runs a fingerprint comparison against the full database, exercising the chemistry code I hand-rolled in assembly more than ten years ago. /predict, the property prediction, takes another 66,000, this time exercising the Python compute backend. The bots are driving the actual machinery, not grazing on static HTML, and it holds up.

This is the part I am quietly happy about. Every year there is a new way to deploy a web application, and every year I am glad I did not rewrite Cheméo to use it. Boring technology, chosen with care, keeps running while you go and do something else.

Almost nobody out there is human

Now the logs themselves. Roughly 99% of the traffic is automated. The real (or direct) human audience is small: somewhere between 50,000 and 100,000 page views a month, call it two to three thousand a day.

You can read it straight off the assets. The main stylesheet, the one a real browser loads once per page, was fetched about 110,000 times a month. That is the generous ceiling on real page renders, because some rendering bots pull the CSS too. About 98% of the so-called page views never asked for a single asset of the page they supposedly loaded. They took the HTML and left.

Some of the bots try to dress up to hide themselves. The traffic carrying a normal browser user-agent averaged 0.67 asset requests per page, where a real browser pulls five to fifteen. It was dominated by a handful of fixed user-agent strings, with 231,519 requests coming from one single Windows and Chrome string. That is not a person. That is a scraper wearing a browser costume, and a bad one!

A few thousand real readers a day, sustained for years with no marketing and no newsletter, is more than fine by me. The site does its job for the people who need a boiling point or a heat of formation.

The part that actually changed

Here is what I find interesting. Split the bots by what they are for:

Who	Share of requests
AI crawlers	37%
Browser-UA (mostly scrapers)	30%
Search engines	20%
SEO and marketing bots	8%
Everything else	5%

The single biggest consumer of Cheméo, by requests and by bandwidth, is no longer a search engine. It is AI training and retrieval crawlers.

AI crawler	Requests in the window	Operator
Amazonbot	257,925	Amazon
ChatGPT-User / OAI-SearchBot	174,846	OpenAI
ClaudeBot	115,945	Anthropic
GPTBot	102,617	OpenAI
Bytespider	36,849	ByteDance

For comparison, the classic search crawlers over the same window were Sogou, Bingbot, Googlebot, YandexBot and Baiduspider. Together they came to about a fifth of the traffic.

I have looked at these logs before, a few years back. Back then the story was search, almost entirely. Googlebot, Bingbot and the rest were something like 80% of the bot traffic, and the AI crawlers did not exist yet.

Today they are the largest single category, and search is down to a fifth. That is the pattern change, and it happened fast. In a few years, the dominant reason a machine visits Cheméo went from "index this page for a search result" to "ingest this data for a model".

What it means for the data

This changes what being found means. For fifteen years, being useful meant ranking in Google so an engineer could land on the right compound page. More and more, the same engineer gets the number straight from a chatbot, and the chatbot got it by crawling Cheméo. The data still comes from here. The reader in the middle is now a model.

So I am doing the obvious thing. I am writing a clear page on where Cheméo's data comes from, how it is collected and validated. Not for SEO, but so the models pulling the data have the provenance right there and can represent and cite it correctly. If the machines are going to read the site more than people do, they should at least get the story right.

I built Cheméo for chemical engineers. Most of its readers are now machines, and the little server in the rack handled the whole switch without me touching it.

Still Faster, After 30 Years

2026-04-18T14:24:00+02:00

Back in the thermodynamic code I keep writing lately, one tiny problem shows up in every single iteration of a flash calculation. You need to solve a cubic equation in the compressibility factor Z, the heart of any cubic equation of state (Peng-Robinson, Soave-Redlich-Kwong, you name it).

A cubic has three roots. Depending on the conditions, you take one or another depending on your goal, if you are looking at the liquid, the vapor or the root minimizing the energy. A simulator calls this solver millions of times per full plant simulation, even more in reservoir simulations. Every nanosecond you save matters.

So, this weekend, I got curious. The thermolib code I have been using for years ships a tight little Newton solver for this cubic, written in Fortran in the 1990s. Is it still the fastest thing in the room, or has an analytical approach caught up in thirty years of compilers and CPUs?

The problem

The cubic we need to solve has a known form:

$Z³ - X₂·Z² + X₁·Z - X₀ = 0$

where $X₀$, $X₁$, $X₂$ come from the reduced EOS parameters $A$, $B$ and the EOS-specific constants (different for PR and SRK, the same for both solvers in my comparison). The physical root is always a bit above $B$ for the liquid phase, or a bit above $B + 1$ for the vapor phase. That is the key domain fact.

Two approaches

The old Fortran way: Newton with Halley (cubic) corrections.

Given the polynomial $f(Z) = Z^3 - X_2 Z^2 + X_1 Z - X_0$ and its first two derivatives $f'$ and $f''$, pick a starting value $Z_0$ by checking the sign of $f$ at the inflection point $Z = X_2 / 3$. Take $Z_0 = B$ on the liquid side ($f > 0$), or $Z_0 = B + 1$ on the vapor side.

Then iterate, with a third-order correction:

$$\delta_n = \frac{f(Z_n)}{f'(Z_n)} \left(1 + \frac{f(Z_n) \, f''(Z_n)}{f'(Z_n)^2}\right), \qquad Z_{n+1} = Z_n - \delta_n$$

Stop when $|\delta_n| < 10^{-7}$.

Two or three iterations and it is done. If a second root is needed, factor the first one out and apply one more Newton step on the remaining quadratic.

The old Cardano/mathematical way: closed form, no iteration. The formula is from 1545, that is not a typo. If you are interested with history, the Wikipedia page about the cubic equation is a rabbit hole you can spent a couple of hours into and then come back here.

Rewrite the cubic as $Z^3 + p Z^2 + q Z + r = 0$ with $p = -X_2$, $q = X_1$, $r = -X_0$, then substitute $Z = t - p/3$ to kill the quadratic term:

$$t^3 + P\,t + Q = 0, \qquad P = q - \frac{p^2}{3}, \qquad Q = \frac{2 p^3}{27} - \frac{p q}{3} + r$$

The discriminant decides which branch to take:

$$\Delta = \left(\frac{Q}{2}\right)^2 + \left(\frac{P}{3}\right)^3$$

If $\Delta > 0$, one real root, given by a sum of two real cube roots:

$$t = \sqrt[3]{-\tfrac{Q}{2} + \sqrt{\Delta}} + \sqrt[3]{-\tfrac{Q}{2} - \sqrt{\Delta}}$$

If $\Delta \leq 0$, three real roots, given by the trigonometric form:

$$t_k = 2 \sqrt{-\tfrac{P}{3}} \, \cos!\left(\frac{1}{3} \arccos!\left(\frac{-Q/2}{\sqrt{-(P/3)^3}}\right) - \frac{2 \pi k}{3}\right), \qquad k = 0, 1, 2$$

Recover $Z = t - p/3$ and pick the physical root.

No loop and no tolerance, a single pass through a 480-year-old formula. On paper, this should be hard to beat.

The benchmark

I reimplemented both solvers in Fortran, in a small standalone file, and tested five cases: single-root, two-phase and dense-liquid regimes. Both agree to machine precision, max error around 2·10⁻¹⁶.

Then I timed them with my benchmarking tools:

Case	Newton	Cardano	Ratio
PR near-critical gas	37.6 ns	45.9 ns	1.22×
PR two-phase region	19.5 ns	46.8 ns	2.40×
PR dense liquid	25.4 ns	45.7 ns	1.80×
SRK low-density gas	22.5 ns	46.5 ns	2.07×
SRK two-phase region	14.5 ns	45.9 ns	3.16×

The old Fortran Newton wins every case. Between 1.22× and 3.16× faster. Cardano stays flat at around 46 ns per call, Newton varies from 14 to 38 ns depending on how close the starting guess was.

Why the old code still wins

Cardano pays for a cbrtf, a sqrt, an acos and a cos on every call. Those are not cheap on a modern CPU. Each takes tens of nanoseconds of hardware work, and the analytical formula needs all of them. There is no shortcut, because the formula is general and does not know anything about where the root should be.

Newton pays for nothing exotic, just a handful of multiplies and adds per iteration. And it converges in two or three iterations, because the starting guess is already close. The author of the Fortran code knew that the physical root is always near $B$ or near $B + 1$. That one piece of domain knowledge is worth a factor of three.

This is the whole point. Cardano is general and cannot skip anything. The Fortran solver is specialised and skips a lot because the author knew where the root lives.

Not a coincidence

The author of that Fortran Newton solver is Michael L. Michelsen, the one from Thermodynamic Models: Fundamentals & Computational Aspects, and the one whose department I was lucky to do my PhD in, in Denmark.

Thirty years later, with a modern Fortran compiler and a modern 2025 (AMD) CPU, I would have expected the general and analytical code to have caught up (maybe through vectorization, better tables, whatever).

It has not. The old code still wins, because it embbeds the domain knowledge of the expert in it.

PS: By the way, a good start Newton/Halley's method implementation can be found on the CD accompanying the book of Michelsen and Mollerup.

The Reference Already on the Shelf

2026-04-11T10:25:00+02:00

These last months, I have been deep amine gas treatement systems. Enjoying playing with eNRTL, the right Henry constants, with all the MEA, DEA, MDEA, piperazine blends, the whole family of solvents that industry uses to strip CO₂ and H₂S (and more) out of process gas. And one reference kept coming back in every paper, every thesis, every validation dataset I was reading: Kohl & Nielsen.

Just "Kohl & Nielsen", like everyone in the field already knows what you mean, but of course, not me.

So I dug a bit. It turned out to be Gas Purification, the 5th edition, by Arthur L. Kohl and Richard B. Nielsen, published nearly 30 years ago, in 1997. The reference on the subject, like Thermodynamic Models: Fundamentals & Computational Aspects by Michelsen and Mollerup for fluid phase equilibria. In it, you get flow schemes, loading correlations, reaction mechanisms, degradation pathways, heats of absorption; all the numbers I need to check my simulations against real plants. On the desk, not on the shelf.

Off I went online looking to buy a used copy of it, I have a collection of used reference books, I am still very old school, still prefer the dead tree version of reference books, because I always put post-its to mark the pages of interest to quickly find them again. Prices looked reasonable, €80 to €150 depending on the seller.

And then I looked at the cover.

The dark blue cover, the "FIFTH EDITION" banner at the top, the not so good photograph of the gas treating plant at the bottom. I had seen this book before. Many times. Too many times. Yes, on my wife's bookshelf, in the office next to mine.

My wife is a chemical engineer and a specialist in gas treating processes. Of course she has the reference work in her field. Of course it is the 5th edition. Of course it was already three meters away from my desk the whole time I was hunting for a copy online.

I raided her bookshelf. A bookmark is now sitting in the MDEA chapter.

The lesson: before you buy the reference book for your domain, check if the domain expert you live with has not already bought it for you.

eIDAS PDF Signatures on Linux

2026-04-05T18:00:00+02:00

Here in Germany, running a GmbH means signing more and more things electronically. The Finanzamt wants e-invoices, contracts move to PAdES, and quite nicely the list of paper-only documents is shrinking. Ok, some times to times, you still need to send a fax or fill a form electronically, which creates a PDF, that you need to sign manually and then post the good old way. Still, the direction is clear: every year, a few more things that used to need a wet-ink signature now need a qualified electronic signature (QES) under eIDAS instead.

So, I got a D-Trust Card 5.1 Std. RSA 2cc and a REINER SCT cyberJack RFID standard reader, and I sat down to make it work with Ubuntu 24.04. I must confess: I had no idea I was about to spend a few evenings reading OpenSC source code.

The card that Linux did not want to talk to

The D-Trust Card 5.1 uses CardOS 6.0, and CardOS 6.0 requires a CAN (Card Access Number) to open a PACE secure channel before the card will answer anything useful. This is a nice security feature — the 6-digit CAN is printed on the card itself, so an attacker with stolen card data but no physical card cannot do much — but it needs support all the way down the stack.

OpenSC did not support D-Trust 5.1 until version 0.27.1, released on 2026-03-31. Before that, pkcs15-tool -D would politely answer "Card is invalid or cannot be handled" and that was the end of the conversation. The implementation landed in OpenSC PR #3137, and the release tracking is in issue #3526. It looks simple, but I had to read the code of OpenSC a bit more than I wanted at first, it was segfaulting on me a bit too much. At the end, I found the bug and created my first PR for this project, happy day.

The lesson is simple: if you are buying a qualified signature card today, check the OpenSC changelog before you open the package. And if you see that you are at the bleeding edge, be ready to spend a bit of time to help iron out the bugs. Once OpenSC was able to talk with my card, the next step was to use it for something useful, that is, sign some PDFs.

What I tried that did not work

The obvious first candidate was pdfsig from poppler-utils, driving the card through NSS. On paper it looks clean: register the OpenSC PKCS#11 module in an NSS database, point pdfsig at it, done. In reality, poppler 24.02 cannot express "no database password, but let the reader collect the card PIN on its own pinpad". Every combination of -nss-pwd '', -nss-pwd 'something', and no flag at all fails. The cyberJack reader has a protected authentication path — PINs must be typed on its physical keypad, not on the laptop — and NSS as used by poppler 24.02 does not forward that properly for this card.

Next candidate: pyHanko with --skip-user-pin. pyHanko has excellent PKCS#11 support, and --skip-user-pin sounds exactly like what you want when the reader handles the PIN itself. Except that --skip-user-pin does not mean "let the token handle the PIN", it means "skip C_Login entirely". Without a successful C_Login, the signing key is invisible (it has the private flag), and pyHanko reports Could not find private key with label 'Signaturzertifikat'. Without the flag, pyHanko prompts for a PIN on the keyboard and pushes it into C_Login, which the card rejects because it expects protected authentication path. Heads you lose, tails you lose.

The working recipe

After a lot of reading of pyhanko/sign/pkcs11.py and python-pkcs11/_pkcs11.pyx, the fix turned out to be a single configuration line.

pyHanko does know how to issue C_Login(CKU_USER, NULL, 0) — the call that tells the token "please handle the PIN yourself". It is exposed through a pin-entry mode called DEFER. The catch: the CLI flags can only reach SKIP or PROMPT. The only way to select DEFER is through a YAML config file loaded with --config.

Here is the minimal pyhanko.yml that makes the whole thing work:

pkcs11-setups:
  smartcard:
    module-path: /usr/lib/opensc-pkcs11.so
    slot-no: 1                      # Signature-PIN slot
    cert-label: Signaturzertifikat
    key-label: Signaturschluessel   # private key label differs from cert label!
    prompt-pin: defer               # the setting that changes everything
    other-certs-to-pull: []
    bulk-fetch: false

And the invocation:

pyhanko --config pyhanko.yml sign addsig \
    --field Sig1 \
    pkcs11 --p11-setup smartcard \
    input.pdf input-signed.pdf

The reader blinks, asks for the CAN on its pinpad, then asks for the Signature-PIN, and out comes a signed PDF. pdfsig input-signed.pdf happily reports Signature is Valid. Total document signed.

Two small details that cost me time:

The certificate label is Signaturzertifikat but the private key label is Signaturschluessel. pyHanko needs both, separately, via cert-label and key-label; using only one of them gives you a very confusing "key not found" error.
The card allows only 3 attempts on the Signature-PIN before it locks. Before running the real sign addsig, write a tiny read-only Python script that opens the token with user_pin=PROTECTED_AUTH and lists the private key labels. A misconfigured run costs you a PIN retry, and you do not want to discover the YAML typo on attempt three.

Adding PAdES and a trusted timestamp

A bare signature is rarely enough for a real invoice. What the EU (and every auditor) expects is a PAdES signature with a trusted RFC 3161 timestamp attached. The good news: pyHanko adds both with two extra flags. The even better news: Sectigo runs a public qualified timestamping authority at http://timestamp.sectigo.com/qualified — no registration, no account, nothing to configure, and it answers in under a second.

pyhanko --config pyhanko.yml sign addsig \
    --field '1/350,40,560,120/Sig1' \
    --style-name default \
    --use-pades \
    --timestamp-url http://timestamp.sectigo.com/qualified \
    pkcs11 --p11-setup smartcard \
    input.pdf input-signed.pdf

The --field syntax PAGE/X1,Y1,X2,Y2/NAME places a visible signature box on the page. A4 is 595×842 points, origin bottom-left, and 350,40,560,120 drops a reasonably sized block in the bottom-right corner of the first page. Use -1 for the last page. And yes, the coordinates must be integers — passing floats gives you a cryptic "Sig field parameters should be four integers" error.

And once you have a signed PDF in your hand, you will want to double-check it against something more authoritative than pdfsig. The EU runs a free signature validation webapp that understands PAdES, walks the whole trust chain, and verifies the timestamp against the EU List of Trusted Lists (LOTL). All green there means a PDF that any auditor in Europe will accept.

What this is worth

At the end, I have a command-line tool that produces qualified, timestamped, PAdES-signed PDFs on Linux, using only open-source software, a €100 reader, and a €150/year card from D-Trust. No virtual machine, no commercial Java signing suite, no "please use Windows", no dependency on a vendor that could decide to discontinue their Linux client next year.

This matters because the list of things that must be signed electronically is only going to grow. Better to have a working, scriptable, reproducible setup on the platform I actually use every day, than to boot a Windows VM every time the Finanzamt invents a new form.

The versions known to work together, as of today:

Ubuntu 24.04.
OpenSC 0.27.1 (built from git, commit c8157c3b6).
pyHanko 0.32.0.
python-pkcs11 0.9.3.
poppler-utils 24.02.0 (for pdfsig — verification only, do not try to sign with it).
D-Trust Card 5.1 Std. RSA 2cc (CardOS 6.0 R1.2).
REINER SCT cyberJack RFID standard.

Enjoy!

A new Logo for Céondo

2026-03-05T22:11:00+01:00

After nearly 20 years of Céondo, time for a refresh, and the refresh started with the logo. I updated the star, it still has the same orientation, but it is now a more conventional star with more structure and a nice 3D feeling using a polygonal shape.

What I like is that it allowed me to have more saturated colors, it is not flat anymore, it kind of feels more alive. Also, because of the structure of the star, it now shows an orientation thanks to the shadow. It was not that easy to get the shadow right to give the feeling that the star is well balanced with the Céondo name and also looking "top right" into the future. At the end, I am pretty pleased.

The next steps will be to update the website, making it ready to enter the 3rd decade of Céondo.

And to remember, especially for me, the old logo:

Sucrosphere

2026-02-07T15:09:00+01:00

The work started in 2023 and developed so well that for the sugar business, I am now exclusively working for Pfeifer & Langen IP GmbH, under the brand name Sucrosphere. Sucrosphere is a spinoff of the 3rd biggest sugar producer in Germany, the 150-year-old Pfeifer & Langen family business. They decided early on, in 2019, to not let the digitalization of the industry overrun them and took a long view, thanks to their family business vision. The long view, maybe a bit crazy at that time, was to develop a complete control system for the sugar industry, from sensors (Vision, NIRS) to model predictive control. Of course, not forgetting scheduling, heat integration, and operator training through a digital twin.

I am happily supporting them as lead modeling expert, bringing my 25 years of experience in the industry to a young and talented team of software developers and engineers backed by years of internal sugar expertise. We have a lot of fun solving hard problems.

For everything else, no exclusivity, so if you need some molecular property expertise, oil & gas, pharma, or fine chemicals modeling, I am there!

TIL - Process Explorer

2023-06-07T17:09:00+02:00

TIL - Today I Learned (Again)

At present, I find myself tackling some rather complex issues related to an Amines gas treatment simulation. There's a certain pleasure in resolving both speciation (we're talking about chemical equilibria using equilibrium constants) and diffusion of the gas that we aim to purify into the fluid phase (this entails chemical reaction rates, discretisation over space and time).

The real issue is that the software my customer provided doesn't quite meet their speed expectations. Having previously built one of the fastest multi-phase, multi-model flashes, my customer requested that I optimize their code.

For numerical work, my solution, as soon as the number of equations starts to be in the hundreds or more, is always to let the big players do the heavy work. In this case, this involves using the MKL for all linear algebra calculations. Having worked with the MKL for over a decade, I usually know exactly which DLLs to package with the application. However, with the introduction of OneAPI, it seems they've decided to play a game of musical chairs with their files - renaming and moving them around, leaving me with some nice C06D007E exceptions.

To identify the necessary DLLs, I built a small application to test all the routines I wanted to use, and then kept tabs on it with the Process Explorer utility. This compact 2MB tool is incredibly useful, providing a host of insights about a running process, such as:

Environment variables
Loaded DLLs
Memory and CPU usage
Number of active threads
Open TCP/IP connections
Level of coffee in your mug
And much more!

With the software running and monitored, I was able to compile the list of necessary DLLs in no time. It was an efficient and hassle-free solution. In fact, it took me more time writing this small note than doing the analysis, so yes, highly recommended -- and you do not need to be administrator to run it!

Cheméo April Updates

2023-05-02T18:54:00+02:00

We improved the API of Cheméo and added more data!

On the data side, we were able to add 200 new mixtures, with a lot of equilibrium data from about 500 mixtures. This was thanks to data collected and generously provided by Davi Girardi, M.Sc., research assistant at the Technical University of Hamburg.
Davi is working on mixed-solvent electrolyte systems based on openCOSMO-RS.
On the API side, we added a new end point /api/v1/cid/{cid}. This allows you to directly retrieve the data of a given compound if you know its ID. This is now for example used by DWSIM to easily retrieve missing data for a simulation.

Again, we welcome your feedback and ideas to improve the database and help you do your data work. Just contact us! As a reminder, you can download the data as Excel, but you can also use the API to pull the data directly into your programs.

Cheméo February Updates

2023-02-27T20:17:00+01:00

More data was what we promised, this is what you get!

On the data side, we were able to add more mixtures, we now have 16067 mixtures in the database. We also improved our data pipeline (the system parsing raw data from publications, performing validation and reconciliation) and this unlocked a large number of pure and mixture datasets.
Vapor pressure correlations are now available in the Excel and PDF exports.
You can bookmarks compounds and mixtures to create a personal worklist. This way, you can easily go back to compounds and mixtures of interest. You can see the attached screenshot of the worklist once a user added a couple mixtures and compounds.
We added a small "ads" for Céondo on the left navigation bar. We think it is not intrusive. If you have better ideas on how to do it, your ideas are welcome.

If you have any questions and remarks, they are welcome! If you like Cheméo, do not hesitate to recommend it to your colleagues, students, etc.

We really want to make a website helping and respecting you, if at any point something is annoying you, please let us know!

Cheméo November Updates

2022-11-30T20:17:00+01:00

Together with Marcus we improved quite a lot the website in the past month. I must say, we were also lucky to have some great feedback from you, end- users. This helped and helps us a lot! Thank you!

Here are the highlights:

again, on your profile page you can delete your account and we will forget everything we know about you. You will not receive any more news from us. We do not track you, we do not set any marketing cookies, etc. This is also why you do not see any "cookie banner" on the website.
we added more data! You have now vapor pressure data in the form of the Antoine and extended Antoine equations. These are coming from KDB and the Yaws Handbook of Vapor Pressure. You can see them if you scroll a bit on, for example, the Octane page (We still need to provide the data in the API, Excel and PDF export) Many thanks to Simon Müller, research associate at the Technological University of Hamburg for providing us with some data, helping us validate some other and providing a lot of feedback! If you need some help with equilibria of electrolyte solutions, I am sure he can help you.
you get mixture data as Excel export.
we improved a lot the navigation and look with a mobile device. The website is now "reactive". I do my research work on a desktop, but I am old, it looks like the new generation is using the website a lot with a mobile. This should make them happy.
we improved the property prediction tool on a slow connection. I am still not very happy with the weight of the molecular drawing tool, I hope to find a simpler one in the next months. Many thanks to Terry Yang, researcher at The Industrial Technology Research Institute in Taiwan for helping us solving this issue.
[fix]: some edge case in the property prediction of molecules with triple bonds. The # was not correctly escaped by the backend and the molecule was "cut" before being sent to the calculation engine.
[fix]: some relationships errors in the database.

What do we have in the pipeline ?

more data, more data and more data.
the ability for you to store private data.

If you have any questions and remarks, they are welcome! If you like Cheméo, do not hesitate to recommend it to your colleagues, students, etc.

We really want to make a website helping and respecting you, if at any point something is annoying you, please let us know!

Major Update of Cheméo

2022-10-11T21:17:00+02:00

It is a great pleasure to announce a major update of Cheméo. The database contains now mixture data for 9016 binary and ternary mixtures. Pure compound data has been increased to 102319 compounds/molecules.

This upgrade would have not been possible without the fantastic work of the ThermoML Archive.

If you register an account, you can get access to an API to query the database or simply download the data as Excel files.

We spent a lot of time to optimize the website, this means that normally, within Europe, every page should load within 200ms (feeling instantaneous). If you are on another side of the planet, the distance to the server is the limiting factor.

Your feedback is welcome!

Céondo and DOME 4.0 for Chemical Properties

2022-08-10T09:02:00+02:00

This is a pleasure to announce that Cheméo is now used by the DOME 4.0 platform. The inclusion of Cheméo as data provider confirms the quality of our work.

DOME 4.0 have released a nice video of the DOME 4.0 platform in action.

The Digital Open Marketplace Ecosystem (DOME) 4.0 offers an industrial data marketplace ecosystem based on Open Science and Open Innovation principles to enable sharing of business-to-business (B2B) data and creation of new or enhanced products, processes and services.

The multi-sided ecosystem DOME 4.0 will be open to all providers as well as users of data and aims to facilitate maximum knowledge extraction with the help of ontology-based semantic data interoperability and modern data processing technologies.

If you think that your chemical property workflow can be improved, do not hesitate to contact us, we are specialized in creating and deploying chemical property databases and analysis tools.

VMware Slow with Ubuntu

2022-01-30T13:42:00+01:00

For each customer requiring some work to be done on a Windows system, I am running a virtual machine. This ensures that I am not leaking information between my customers and allows me to have just the right software stack installed for each of them.

Last year, I upgraded from Ubuntu 18.04 to 20.04. I am very conservative in my base operating system, this is my work horse, I cannot allow myself to lose data or work with a broken system. This is why I take my time before I upgrade.

At the end, the upgrade was a good thing, except for one thing.

Even if I upgraded VMWare Workstation from 15 to 16, it was slow like hell.

To solve this issue, I had to try so many things that I cannot be sure which one really helped or if this is a combination of all of them which at the end restored correct performance for the system.

3D Acceleration

It looks like VMWare Workstation 16 has issues with the 3D acceleration (I have an Intel integrated GPU on my laptop). I disabled it in the options of the VM. Note that the latest point release of VMware Workstation solved this issue.

Removing Mitigations

At the end, I remembered that with the newer kernels, the mitigations against all the security issues in the Intel CPU were enabled by default. I disabled them to regain a normally efficient laptop. My next laptop will not run with an Intel CPU.

How to disable the mitigations?

Very simple, edit as administrator the file /etc/default/grub. Find the line with:

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"

and replace it with:

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash mitigations=off"

Then run in the console/terminal:

$ sudo update-grub

And reboot your system.

You now have an unsecure system with respect to the Spectre class of attacks (and some more), but in my case a bit more usable system.

This is not recommended if you run untrusted code on your system. In my case, the less trusted code is running in a virtual machine, it means the code could use side channel attacks to gain knowledge of the main host software, but this is acceptable for my workload.

Switching to Wayland

The last action was to start Gnome not with xorg but with Wayland. Do not ask me to explain the details between Wayland and xorg, but using it (you select the one you want on the login screen) resulted in a generally very snappy desktop. I wonder if this change only could be the only one needed. Once I have the time, I may experiment a bit.

Update: After many trials, Wayland has too many disadvantages, like missing screensharing in MS Teams or special dialogs in some of my applications not showing correctly. I went back to Xorg. I am still not sure how what the best way is.

Latest update: It looks like VMWare 16.2 has now a correct support of 3D acceleration on Intel GPU, this should solve the issues.

Asynchronous IO

As it did not really solved the issue, I started looking at the vmware.log in more details. If found this line:

2021-11-16T09:43:19.544Z Wa(03) vmx Couldn't initialize aioKernel: dlopen failed for \
    libaio.so.1: libaio.so.1: cannot open shared object file: No such file or directory

I removed it by installing libaio:

sudo apt-get install libaio1

We will see if it improves the situation.

Type of Disk

The issue was still not solved. The last trial is to change the type of disk from SCSI to NVMe. For the moment, I am trying only on one VM. I had first to clone the full VM (not a linked clone, a full copy) and then create the custom VM with the source disk being the disk of the clone. It does not create again another disk for the conversion, so you end up with 3 copies of the data, only two.

Kernel Memory Management

It looks like I was able to find something effectively improving the situation, by adding these two lines to my /etc/sysctl.conf file:

vm.compaction_proactiveness=0
vm.swappiness = 5

For reference, the kernel is a 5.13:

$ uname -a
Linux host 5.13.0-28-generic

Welcome Marcus!

2021-11-29T21:29:00+01:00

Since 2007, Céondo has been a one man show.

I have been lucky to have only very nice customers, all great engineers and passionate about their work. They allowed and allow me to not only provide them great services but also helped me to grow and improve. I would not be here without them.

And thanks to their repeated trust in my work, Céondo is growing from a one man show, to a tiny team of two.

Since this November, I am really thrilled to work together with Marcus.

Marcus has a Master’s degree in Chemical Engineering. He is passionate about processing engineering, numerical methods, software design and software development, having particular interest in developing well-thought and maintainable software and libraries. He is from Minas Gerais, Brazil and has a good knowledge of many programming languages, especially Python and modern C++.

Marcus has done and keeps doing great work for CO-LaN. He will of course continue but will also be the driving force behind the new release of Cheméo.

Timing a Fortran Program

2020-10-27T10:29:00+01:00

This is a bit of Fortran you find in many simulations:

DO I=1,NVALS
    IF (RNDVALS(I).GT.CUT) THEN
        DTMP = RNDVALS(I)
    ELSE
        DTMP = CUT
    END IF
    LOGVALS(I) = LOG(DTMP)
END DO

Or without the saving in the temporary DTMP variable. This does not look very nice, you have basically 5 lines to control the value before proceeding with calculations. An alternative is to push everything into the MAX intrinsic function:

DO I=1,NVALS
    LOGVALS(I) = LOG(MAX(RNDVALS(I),CUT))
END DO

This make the code cleaner, but it costs a MAX call at each iteration. The first rule in programming is that if you want to go fast, you need to do less. As I do not want to make my code significantly slower, I decided to benchmark the two different versions. So, I simulated the two cases with this sample code with calls to CPU_TIME to track the start/stop and a long enough run to gather reproducible values:

PROGRAM TEST

INTEGER, PARAMETER :: NVALS = 1000000
DOUBLE PRECISION RNDVALS(NVALS)
DOUBLE PRECISION LOGVALS(NVALS)
INTEGER I, J
INTEGER, PARAMETER :: NRUNS = 100
DOUBLE PRECISION, PARAMETER :: CUT = 0.001D0
DOUBLE PRECISION :: START_TA, STOP_TA, START_TB, STOP_TB
DOUBLE PRECISION DTMP, A, B

CALL RANDOM_NUMBER(RNDVALS)

CALL CPU_TIME(START_TA)
DO J=1, NRUNS
    DO I=1,NVALS
        IF (RNDVALS(I).GT.CUT) THEN
           LOGVALS(I) = LOG(RNDVALS(I))
        ELSE
           LOGVALS(I) = LOG(CUT)
        END IF
    END DO
END DO
CALL CPU_TIME(STOP_TA)

CALL CPU_TIME(START_TB)
DO J=1, NRUNS
    DO I=1,NVALS
        LOGVALS(I) = LOG(MAX(RNDVALS(I),CUT))
    END DO
END DO
CALL CPU_TIME(STOP_TB)

WRITE(*,*) '....', LOGVALS(143), '...'

A = STOP_TA - START_TA
B = STOP_TB - START_TB
WRITE(*,*) 'IF ELSE', A
WRITE(*,*) 'MAX    ', B
WRITE(*,*) 'DIFF % ', (A-B)/A * 100.0D0

END PROGRAM

I compiled this test program with:

gfortran  -Ofast -funroll-loops test.for

I did some runs with the IF/ELSE first and with the MAX first and basically, the results are the same. But a significant difference is coming from the CUT parameter.

CUT is at 0.5D0, the IF/ELSE approach is 40% faster.
CUT is at 0.001D0, the MAX approach is 2% faster.

Yes, the cut is the performance control. This is most likely because of the branch prediction. MAX is a function call and it costs most likely more to recover after a wrong prediction. Because the array if just full of random values between 0 and 1, in 50% of the cases, the prediction is false if the cut is at 0.5. If the cut is lower, that is the MAX is nearly always returning the MAX of one of the two values and always the same, it is faster.

Because in my thermodynamic code, the control on the value is the exceptional case, that is, we normally never trigger the CUT, it means that the branch prediction will be nearly always good and the MAX approach is not only as fast as the IF/ELSE case but nicer to read.

I was expecting the MAX call to always be slower.

Complex Flash Calculations

2020-08-21T10:29:00+02:00

In this simulation, we took special care not to harm any chemical compounds. But, to protect their identities, we replaced their names. Ok, practically, COMP1 to COMP13 are the usual suspect from Methane to Nonane with some gases like maybe CO2 or others. The PETRO_1 to PETRO_11 are petro cuts. H20 is left because it is really obvious. If you see - as fraction for a phase, this is because the fraction is lower than 1.0E-35. The flash can find theoritical fractions way below that. This not that important for a standard flash, if your fraction is 1.0E-15 or 1.0E-45, you do not really care, but this is important when you are looking at phase boundaries.

Anyway, this is a beautiful flash problem. If you look at the two liquid phases, they are nearly identical. I am really proud that my flash code is able to solve such a hard problem and not lose the second liquid.

-------------------------------------------------------------------------- 
               P(Pa)  31455470.00      P(bar)      314.55
                T(K)       299.82      T(cel)       26.67
               MODEL         1         1         1         1 
                   Z  1.374942  1.262459  0.969510  0.300845 
               STATE   LIQUID    LIQUID    VAPOR     LIQUID  
              NATURE  HYDROCAR  HYDROCAR  HYDROCAR  AQUEOUS  
                BETA  0.017572  0.007757  0.469552  0.505119  FEED
-------------------------------------------------------------------------- 
PETRO_1a              0.088650  0.076813  0.020270     -      0.011672 
PETRO_2               0.002647  0.002358  0.000830  1.92E-35  0.000455 
PETRO_3               0.003136  0.002713  0.000731     -      0.000420 
PETRO_4               0.002137  0.001806  0.000393     -      0.000236 
PETRO_6               0.003024  0.002497  0.000441     -      0.000280 
PETRO_7               0.003148  0.002452  0.000270     -      0.000201 
PETRO_8               0.003845  0.002763  0.000164     -      0.000166 
PETRO_9               0.004654  0.002991  0.000074     -      0.000140 
PETRO_10              0.004411  0.002434  0.000018     -      0.000105 
PETRO_11              0.003687  0.001648  2.38E-06     -      0.000079 
COMP1                 0.002344  0.002528  0.003821  5.63E-09  0.001855 
COMP2                 0.004319  0.004378  0.004518  1.06E-06  0.002232 
COMP3                 0.594647  0.621946  0.761958  6.10E-07  0.373053 
COMP4                 0.064187  0.065363  0.065147  1.48E-10  0.032225 
COMP5                 0.057369  0.057435  0.049935  2.08E-13  0.024901 
COMP6                 0.033973  0.033108  0.024641  1.44E-16  0.012424 
COMP7                 0.013206  0.012940  0.009982  4.84E-17  0.005019 
COMP8                 0.010395  0.010115  0.007215  9.29E-20  0.003649 
COMP9                 0.015588  0.014942  0.009813  6.80E-20  0.004997 
COMP10                0.015740  0.014890  0.008625  2.60E-23  0.004442 
COMP11                0.026786  0.025126  0.013396  1.24E-26  0.006956 
COMP12                0.025084  0.023153  0.010866  1.95E-30  0.005723 
COMP13                0.016487  0.015099  0.006511  1.99E-34  0.003464 
H2O                   0.000537  0.000502  0.000377  0.999998  0.505309 
--------------------------------------------------------------------------

As a small reminder of the flash algorithm, you basically solve the flash at a fixed number of phases starting from a good initialization and using the stability analysis, you check if you need to introduce a new phase or not. The introduction of a new phase is done if we find a phase which introduced in a very small amount will reduce the Gibbs energy of the system. Here we introduce not only a phase which reduces by a very small amount the Gibbs energy of the system (G/RT = 1x10^-18 where normally we are in the order of 1x10^-4 to 1) but which is also really close to an existing phase. Because it is really close to an existing phase, it is very easy for the solution to converge to the trivial solution of the existing phase. And what you cannot see, is that after one of the stability calls, the convergence needs to overcome a local minima barrier to reach the good minimum and convergence.

So, today I am happy. This is the kind of work I am really proud to do for my customers.

Invalid Pointer in Fortran

2020-08-14T16:29:00+02:00

First, I really enjoy working with Fortran as a language. The relative simplicity and close to the metal of this language makes it very easy to write insanely efficient code. Fortran is old but also very modern thanks to regular updates of the standard and its very good support by Intel and GCC. One beautiful thing is that you can keep coding like it was in the 90's, very performance oriented and use just a bit of the new features to manage more easily large amount of data. That is, you can create user defined data types to store stuff:

TYPE MySmallStuff
    INTEGER          :: foo   !< The ultimate foo value
    DOUBLE PRECISION,ALLOCATABLE:: values(:)      !< Some values 
END TYPE

TYPE MyBigStuff
    INTEGER          :: nsmalls
    TYPE(MySmallStuff),ALLOCATABLE:: smalls(:)   
END TYPE

What you have here, one base derived data type, MyBigStuff storing a series of MySmallStuff. In my case, it was the configuration of an optimizer storing a series of parameters to be optimized. The values for each parameter was the value at each iteration of each parameter.

When using my code, I had sometimes the following error:

free(): invalid pointer

Program received signal SIGABRT: Process abort signal.

Backtrace for this error:
#0  0x7fa42a9ee2ed in ???
#1  0x7fa42a9ed503 in ???
#2  0x7fa42a282fcf in ???
#3  0x7fa42a282f47 in ???
#4  0x7fa42a2848b0 in ???

What was interesting is that the error was when trying to deallocate (automatically or not) on of the elements in smalls. The reason was not at all what I expected: I started to write at indice 0 in values of MySmallStuff.

After writing the historical values, I was able to print them correctly, iterating on values(1:nvalues) but freeing the same allocatable I was able to read values from was crashing with the invalid pointer. This was because I allocated MyBigStuff%smalls(2), then for each smalls, I allocated the values. When writing for one of them at values(0), I destroyed part of the pointer information. This is what triggered the error. Normally I run my code with boundary checks on the arrays, but not in this particular case. I suppose the boundary checks would have caught the off-by-one error...

I was taken off guard because normally, errors in Fortran are pretty obvious, this one took me nearly one hour to pinpoint.

Middle of the Year 2020

2020-07-02T11:01:00+02:00

After a bit more than three months under this strange situation, I can only say that I am lucky. The business is still running smoothly. I expect the turnover to be 10 to 15% below what was planned, but not because of lack of work. This will be mostly the effects of the thee months with home schooling and work from home. In such conditions, it is harder to work as well as usual, this reduced the number of hours I could achieved for my customers. The school is supposed to start again normally in August, but I prefer to plan with again, another 5 months with reduced productivity. We will see.

Now, during my usual look for interesting stuff in modelling, I found that ABB is using Modelica to perform the optimisation of distributed energy production. This is really interesting, they can control biogas based power production, including wind and solar energy. The grid is getting integrated to be more efficient. I wonder if they are going to coordinate with providers to link in the system all the home based solar production. Now that batteries are really cheap, it would be interesting to allow a quick release/store of about 1% of the capacity of the at home deployed batteries to support the stability of the grid.

Corona Virus, Home Office

2020-03-14T11:05:00+01:00

Nothing particular with respect to the global situation. Effective today, I am not going to travel anywhere until middle of April. Next week will most likely see a drop in my efficiency which I expect to recover rapidly. This is because all the schools, all the external group activities are stopped. It is recommended for all the workers of BP refinery offices to work from home.

We will have to manage work and home schooling for the complete family for 5 weeks. At the european scale, this is turning into an interesting social experiment. I suppose we are going to have a lot of papers in psychology journals 6 months down the line...

Update 2020.03.21: After a week, the family ist still together and it went better than expected. Only two days (Monday and Friday) were below my work average. I am confident that with a bit more organisation and spreading my work also over the week-ends I will be able to keep the ball rolling.

Meanwhile, I would like to thank you, my customers, for the very nice words and your support during these difficult times. This pandemic is going to reshape our societies, can we take the opportunity to rethink our economy, the place of the environment, the single point of failure of our global supply chain with China and simply our relationship with our neighbours, the way we live together within a city? These are going to be interesting times.

Update 2020.06.12: Last week, I was able to have my first business trip again. It was in France, my usual hotel was not yet open, the restaurants are running mostly on reduced offers, but the life is slowly going back to normality. I expect flying to be a bit constrained for some months with less flights and more video conferences. For the daily life, we will see the statistics after the reopening of the primary schools on Monday. The primary schools will be fully open for 2 weeks before the summer vacations, this impulse in the system will provide us some nice statistics down the line and if this creates real issues, the 6 week long summer vacations closure will damp the effects.

Different MKL Version for a DLL in Fortran

2020-02-20T10:05:00+01:00

A part of my job is programming special unit operations in PRO/II. These unit operations are delivered as a DLL (UASLB.dll) that is put in the folder of the PRO/II executables. The interface with the PRO/II internals is very simple, it is simple Fortran with a series of functions, subroutines and old school COMMON blocks. The recommended compiler to create the DLL is the well known Intel Fortran compiler. Under Windows, from there, everything is simple, you have some sample code that you can modify, you compile, put the DLL where it should be and launch PRO/II to test.

But... if you want to use the Maths Kernel Library, then you have a problem. Why? Because PRO/II is itself using the MKL and as such, you need to build and deliver your code with the same version of the MKL. This is pretty annoying because you are suddenly locked and cannot use another version of the MKL and sometimes you cannot get the exact same version, for example, depending on the state of your Intel Fortran license.

The solution is partially given in this question on StackOverflow. This is the method for some C++ code, but we can achieve the same in Fortran.

Create a text file, named Intel.MKL.manifest (without the .txt extension) with the following content:

<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">
    <assemblyIdentity name="Intel.MKL" processorArchitecture="amd64" version="2019.0.5.1" type="win32" />
    <file name="mkl_core.dll" />
    <file name="mkl_sequential.dll" />
</assembly>

The version string must match the version string of the mkl_core.dll you find in your redistribution folder of the MKL. For my installation, the folder is: C://Program Files (x86)/IntelSWTools/compilers_and_libraries_2019/windows/redist/intel64_win/mkl. Please note that in my case, I am running everything as 64-Bit versions, but you can have x86 instead of amd64. In fact, the processorArchitecture key is optional.

Given the bin folder where your DLL, here UASLB.dll, is, create the subfolder Intel.MKL in it.
In the Intel.MKL folder, copy the content of the redistribution of the MKL in it. That is, you should have mkl_core.dll, mkl_sequential.dll, etc. in the bin/Intel.MKL folder.
Put, like said in the SO question, the Intel.MKL.manifest file inside the Intel.MKL folder.

The rest of the work has to be done at the creation of the DLL by embedding a manifest inside the DLL referencing the Intel.MKL assembly with its right version. Fortran is not C++, so the best way to reference our special assembly is to do it in the options of the project.

Enable the generation of the manifest.
Add in the command line of the linker the following argument: /manifestDependency:"name='Intel.MKL' processorArchitecture='amd64' version='2019.0.5.1' type='win32'"

During compilation, it will create in the output folder an .intermediate.manifest file which should have the following content:

<?xml version='1.0' encoding='UTF-8' standalone='yes'?>
<assembly xmlns='urn:schemas-microsoft-com:asm.v1' manifestVersion='1.0'>
  <trustInfo xmlns="urn:schemas-microsoft-com:asm.v3">
    <security>
      <requestedPrivileges>
        <requestedExecutionLevel level='asInvoker' uiAccess='false' />
      </requestedPrivileges>
    </security>
  </trustInfo>
  <dependency>
    <dependentAssembly>
      <assemblyIdentity name='Intel.MKL' processorArchitecture='amd64' version='2019.0.5.1' type='win32' />
    </dependentAssembly>
  </dependency>
</assembly>

This is what is then embedded in the DLL. The result is that the DLL will correctly load the your MKL version. You can see it with a dependency walker. Please note that all these manifest stuff is the Windows way of loading the right library and this is working better than expected.

Do not hesitate to contact me if you have Fortran questions.

Visual Studio Code for Fortran

2018-09-19T16:29:00+02:00

Sometimes one needs to say things one never imagined one would say:

Thank you Microsoft for the code editor I am using under Linux!

Yes, all these words in the same sentence! Microsoft in the past years started and released a wonderful code editor, the name is Visual Studio Code. The naming was a bit annoying at first because we have Visual Studio which is a totally different software, but I suppose that on the long run they want the name Visual Studio to be associated with everything they do for the developers more than a specific software.

But what brought me to use some Microsoft software under Linux? Simple productivity. My goal is to always be extremely productive and I am always, on a regular basis, evaluating the different tools to assess if they can provide after the associated learning curve a net benefit for me and my customers. I am paid by the hour in my work and I try to have such highly efficient hours that my customers never complain about the bill (never had one complain in 10 years, keep crossing my fingers).

So, after a bit of wondering, I decided to test Visual Studio Code even so it has at first sight some issues:

It consumes a lot of resources, from 30MB for my Emacs session to more than 400MB for the same code.
It is written in JavaScript and the graphical interface is basically done in HTML/CSS which is not the kind of stack one expect to be efficient for a text editor.
It is written by Microsoft, which is not known to support Linux user very well.

But productivity is more important than 500MB RAM on my system which anyway is equipped with 16GB, Microsoft is supposed to have solve the latency issue of the rendering of the text editor to end-up with something nice and it is always better not to be dogmatic and look what the other side is doing (and Visual Studio has long been a wonderful piece of software).

All things said, I decided to try it and after a bit of adjustment here is the view of the beast when I am working on some Fortran code:

You have basically everything you need:

nice syntax highlighting with the fortran extension;
intellisense with the fortran language server;
git integration;
debugger integration;
and much more because you can easily have many different languages in the same workspace.

Ease of Extension

Microsoft is really showing its expertise at developing developer software with it and also their fresh look at what others have been doing for a long time, that is, the concept of extensions pioneered by Mozilla for Firefox.

This is because all the power of Visual Studio Code is coming from a very well though system of extensions. With a couple of key strokes, you can install and use a large number of extensions. You an extension for everything you may need.

They also pushed ahead the Language Server Protocol (LSP) to integrate all the IntelliSense functionalities of any kind of software into the editor. The LSP is an open standard and it is relatively easy to develop a server. Because the servers providing the language support are independent of the editor, they can be used by any software (yes, Emacs/Vim too).

For example, I was able to quickly improve the one for Fortran here and there.

The extension for the syntax highlighting is always pretty simple and I was able to fix the support of comments in fixed format mode.

All that within a week of using the software...

But Still Place for Improvements

Navigation Within a File

I am still missing some of the great navigation control of Emacs as VS code is using a lot the arrow keys to navigate. I do prefer the Ctrl + key approach from Emacs and I have already added some custom keybindings to help me. This is still a work in progress.

The reason why prefer the navigation using Ctrl + key is because I need to move my hands to access the arrow keys on my Typematrix 2030. I can already feel that I am using a bit too much the mouse because of the non intuitive navigation.

Fixed Format Fortran (Punchcard)

The good old fixed format Fortran is still a big part of my codebase, this is why I need a good support. What I am missing from Emacs is the really good automatic indentation, ease of line breaking with continuation character and automatic change of color of the code after the column 72. I have a partial alternative with some user settings to set a rulers at different columns and force tabs as 3 space characters:

"[fortran]": {
    "editor.rulers": [
        6,
        72,
        80
    ],
    "editor.tabSize": 3,
    "editor.insertSpaces": true,
    "editor.detectIndentation": false
},
"workbench.colorCustomizations": {
    "editorRuler.foreground": "#ebebeb"
},

The current Fortran extension working with the language server is not as good as the modern Fortran extension. But the language server is a need to have all the great contextual help. So, I will need to see if I can get the language server to be supported in this better extension.

Conclusion?

For the moment, my conclusion is simple: I really like it and it is really improving my efficiency at work, this is why I really recommend you to try it!

Update

To solve the missing continuation line command, using the macros extension, I created the following macro (defined in my settings.json):

"macros": {
    "fortranContinuation": [
        {
            "command": "type",
            "args": {
                "text": "\n"
            }
        },
        "deleteAllLeft",
        {
            "command": "type",
            "args": {
                "text": "     $"
            }
        },
    ]
},

And then in my keybindings.json I added:

{
    "key": "ctrl+shift+j",
    "command": "macros.fortranContinuation",
    "when": "editorTextFocus && editorLangId == 'fortran_fixed-form'"
}

this enables the binding only for the fortran_fixed-form language. This is the language id of the fixed format Fortran from the modern Fortran extension.

To use the modern Fortran together with the Fortran Language Server, I had to patch the LS glue extension and the modern Fortran extension. The two changes are basically just to ensure that the extensions are enabled at the same time. In my settings.json I changed the key "[fortran]" with "[fortran_fixed-form]".

For the navigation, the emacs key bindings extensions were not convincing. I will need to explore a bit more the different options, especially the ability to have key bindings enabled only within different contexts.

Update 2

Am currently trying an improved Emacs extension which is working pretty well. We will see how it goes over the long run.
I created a fork of the language server including a series of fixes, you can install the master branch to have all the fixes already applied:
I still have to fork the corresponding VSCode extensions to nicely package the patches I made and allow integration.

10 Years of Home Office, Lessons Learned

2018-02-15T15:29:00+01:00

Since 2005, I am mostly working from my home office, both as an employee for about 2 years and as a consultant for 10 years. The tips are mostly from my work as a consultant but some of them are also very useful as an employee.

Core Principles

Simplicity and sustainability are the principles at the core and everything flow out of them.

Sustainable Work

Of course, your work must be sustainable from a business point of view, but here I mean from a physical point of view. It seems that in the US, it is common practice to work 70h a week and take 3 weeks of vacations a year but reading your work emails 365 days a year. If you want to be a long term consultant, this is a very good recipe to be a bad consultant. Above 40h a week of normal work, the quality of each extra hour decreases rapidly to the point of doing negative work. So, sleep well, do not work long hours and exercise enough.

Bad sleep (in my case, I need 7 to 8h) results in bad work. The worse is that in most of the cases, you feel a bit tired but you are unable to see how your productivity is decreasing. For me, I have a simple way to know if I am tired without knowing it or not: do some fluid phase equilibria and thermodynamics work in Fortran. This is so demanding that if tired I simply cannot do it correctly. If you ask me to code within the .NET or Python ecosystem, the editor and the tools are all there to help you and the lack of "clear" mind is less obvious. Note that this is a vicious circle, because as soon as you are tired you are losing your judgement to evaluate how much your productivity is decreasing.

For the exercise I am guilty of not doing enough of it. About 10 minutes a day of push-ups etc. but I should really have 2h of cycling a week on top of that.

Simplicity

Keep things simple. Simplicity is hard to achieve because you can only reach it when you know what you need and what is optional. You get to know that with your experience and experience is a nice word used by older people to say: "I already made all the mistakes before but you also need to go through to understand the stuff."

You do not have to worry about the future, the natural entropy of your work system will lead to a natural increase in the complexity of it, this is why you do not need to make it more complex that necessary to start with.

Practical Tips

Time Tracking

You simply cannot do without an easy to use time tracking application. After the years I found that the only requirements are:

offline usage (you do not want to track manually the time if you do not have internet connectivity like in a plane or somewhere far away from the civilization)
easy export of the data as CSV or XML to generate custom reports, like the ones you will have to generate to invoice your customers
tagging of the entries to associate them to a project or a customer, a standard tag for a customer entry is the customer name and "Invoiced" if this is invoiced time. This allows me to report at the end of the months all my invoiced hours or only the one for a given customer.

For me a very efficient simple and flexible time tracking tool is the Hamster application in Gnome with the Gnome Applet. At the start, I was using a simple Excel sheet. But now, I can generate my monthly run by day, my worklog per customer, simple and efficient (it has a Python API).

Here is for example an example of a customer page. You can see that I provide the full log, directly out of my time tracking application. This is simple and very efficient. The page is built from an XML export of my hamster time tracking.

Get Used to Work like Athlete Train and Compete

If you ask an athlete how it feels when she trained to hard the day before, the answer is very simple: bad. An athlete tries to be at peak conditions all the time outside of the planned recovery periods.

As an excellent consultant, you want to provide your best each time you work for a customer, it means that overdoing one day automatically implies the need to recover afterwards. After an event, athletes take the time to recover. After a bit of crunch time, you need to recover.

Contrary to the athletes who are doing a lot of training/competition/recovery cycles in the year, as a consultant you try to smooth as much as possible your workload to reduce the number of crunch time/recovery cycles.

This is a relaxed marathon, not a sprint, so ensure that if you need to work 200h over 10 weeks for a customer, you are effectively working 4 to 5h a day. You must not work 70h a week the last 3 weeks!

As you can see on an example view of a monthly load tracking, if I am pushing my limits for a customer on a given day, I ensure to recover immediately thereafter. Of course, the default rule is not to work during the weekend.

Low Working Hours per Customer and per Day

The corollary of running your marathon is that you can keep the number of working hours per customer per day low. I try to always be below 5h of work per customer and per day. If I focus 4 to 5h on a given subject for the day, I can switch to another for 2 to 3h or so and keep 7h of good invoiceable work. My efficiency is lower if I do 7h for a customer a day and 7h for the other the next day. But more than 2 customers/type of work during the day (basically, going to small chunks of work instead of bigger ones) affects badly my productivity.

My ideal run is 2x2h in the morning and 2.5h in the afternoon, split over two customers. This is also why if setup a project for a customer for 10 weeks of work, I always schedule it on a period of 20 weeks. On the long run, this provides extremely high quality hours for my customers.

Simple Planning

Very simple TODO list. Working from home is hard because everything in the house can bring you out of your work. A simple todo list to just write down what needs to be done at home if it goes through my mind frees me to keep working. I do not need to keep in mind that I need to do X or Y after my work. It is on the list. The list is a simple paper list, yes a simple A4 paper on my desk. I have two of them, one for the at home stuff I do not want to pollute my mind with while working and one for the work.

Rough weekly planning. This is just to know when something needs to be done, the hard deadlines of the week, etc. Again, just a piece of paper for it to be simple and reduce the cognitive load of making it. Some weeks, it is nearly empty and this is good so.

This simple planning extends very well for your turnover. Suppose you want to achieve a €100k turnover per year and you charge 100€/h of work with 42 weeks of invoiceable time per year. This means 1000h of work spread over 210 days or 4h45 minutes of invoiced work per day.

So, if every single day you work, you put first these 5h of work for your customers, you do not have to worry, at the end of the year, you will break even and be happy.

Some Other Tips

Other things which helped me:

Be sure to develop the habit of doing every single day some great work for your customers.
Kill distractions during the working hours like a firewall preventing you to access social medias, news, etc.
Keep your desk clean to keep your mind clutter free.
Go to zero emails in your Inbox.

Bonne Année 2018

2018-01-10T16:05:00+01:00

Now that you are all back at work, this is the perfect time to wish everybody all the best for 2018. You can read my best wishes for 2018.

Christmas Holidays

2017-12-22T14:07:00+01:00

Thank you for the very nice year working with many people doing great work. I am taking a break until the first week of January. During the first week of January I will have only reduced availability but you should be able to reach me per mail.

Please note that I really need a break, it means that only urgent matter will be handled. I will take care of everything none urgent when back at the office full time.

Glyphosate and Monsanto's Roundup

2017-08-30T15:34:00+02:00

Today, the French Ministry of Environment announced that they will vote against the renewal of the market authorization of the Glyphosate in the EU. The Glyphosate is a very controversial molecule because, sold under the Roundup it embeds everything the ecological mouvement despises:

the RoundUp is used in combination with GMOs.
the RoundUp is sold and controled by a very powerful company.
the RoundUp combined with the GMOs leads to an increased used of pesticides.

So, basically, Glyphosate is now in the mind of people totally associated with RoundUp and as RoundUp is extremely negatively charged in the public opinion, it starts to be hard to have a rational thinking about this particular molecule.

And you know what? It could be that Glyphosate is a really safe molecule while RoundUp is the most toxic mixture you can buy.

And this is the problem we have right now. That is, Monsanto is maybe totally right when telling that Glyphosate is by itself safe. The real problem is that you never use Glyphosate alone on your field to take out the weeds. If you want to hard kill everything with the RoundUp, you end up with a solution containing about:

0.70%: Glyphosate
0.70%: Other chemicals
98.60%: Water

This is what I read from the usage instructions of the Roundup® Weed & Grass Killer Super Concentrate. That you have so much water in the solution you spray is normal, nowadays, a good active ingredient (pesticide, fungicide) is spread at a rate of 12 to 25g per hectare (10,000 square meters). But to achieve a good mixing you package these minimal amounts in a formulation, this formulation will be then mixed with water in a big tank and this will be spread.

With a bit of simplification you follow the sequence:

10g of active ingredient,
put into 1L of formulation,
mixed into a 1000L of Water solution ready to use.

The critical step is in fact the second one, you can have perfectly good active ingredient and makes a totally toxic formulation. A big part of the work done by the agrochemical companies developing new pesticides/fungicides/herbicides is to ensure that they do not get a toxic formulation.

Because the active ingredient is just a part of what is effectively used on the fields, the validation should not focus on the official active ingredient but on the formulation. If one just focus on the main active ingredient, it opens the door to creation of formulations which are maybe way more effective to kill the weeds, but terrible for our environment.

Update: Some people asked me why mixing a safe compound into a given formulation could make the whole stuff toxic. This is simply because the other compounds in the formulation can act as a carrier and help transport the relatively safe chemical in parts of the organism where it can be toxic. This also why you should never mix prescription drugs with other over-the-counter drugs you can freely buy. Your doctor is providing you a given prescription considering you do not take other things together, if you add some extra stuff, it could have bad side effects. This resulted in medicine in the empirical rule not to give more than 3 drugs to a patient.

Using Yubikeys for our System Security

2017-05-09T12:56:00+02:00

Security is a moving target. Just a couple of days ago, in the German newspaper, we read that the mTan system to secure banking transaction was circumvent allowing attackers to transfer money without authorization. What was considered as secured was not anymore.

What is really interesting nowadays is that we moved away from a local threat to a remote threat. You are now more likely to suffer data loss from a remote bad actor than from someone breaking into your office or house.

To model the threat for the security of Céondo's systems, we need a trusted and secure root. For us, it is pretty simple, it is a safe at the bank. The safe is only accessible with a physical key, a bank card and a PIN associated to the bank card. The PIN is not the same as the normal PIN of the card. The bank is the key and document backup root location, it also the place where we keep a rotation of encrypted backup hard drives.

At the office, the laptops are secured this way:

encrypted home directory (with Ubuntu OS);
login password composed of 8 characters remembered by the user plus a 30+ long password stored as static password on a Yubikey.

On each laptop, a KeePassX vault is used to store all the login/passwords of the different services. All the passwords are unique per service and randomly generated with KeePassX.

The vault itself is encrypted using the same approach as the login, with passphrase composed of normal remembered password and then a long password stored on a Yubikey. Some have the same key for the login and the vault, some have two different keys.

The combination of a remembered password and a long password from the keys for the static passwords allows good resistance to the local threat, that is, someone stealing a laptop with the key on it (as long as the remembered password is not 1234 of course).

For the highly critical accounts, a simple login/password, even randomly generated is not enough. For them, we use another authentication factor. This can be OTP with Gandi or U2F for FastMail.

For each key, we keep a backup key at the bank. Only few services are using OTP/U2F, so for the moment this is not a problem to go to the bank and pull the backup key the time to register it. This may need to be changed as U2F is more widely used.

Using this approach, our systems are well secured and they are still easy to use in the daily life which is really important.

Happy Birthday Céondo

2017-05-08T12:19:00+02:00

In Spring 2007, I decided to start Céondo, at that time I had some ideas about the things I wanted to do, but was not totally sure about them. A lot of unknowns, this is why the company name Céondo was generated using a simple algorithm to be short, without particular meaning, easy to pronounce even for English speakers and with the corresponding .com available.

The company started as a UK limited and in 2014 jumped the channel and is now a German GmbH. Consultancy is a funny ride with highs and lows, but 2017 is a lucky nice year already scheduled to be the best year to date for Céondo. For a 10 year anniversary, this is great!

Thank you to all my customers who made it possible for Céondo to be here today and I am ready to another 10 years like that!

New RAM for a Lenovo X220

2017-02-16T13:29:00+01:00

In the past few weeks, my laptop suffered from very annoying random restarts. Randomly and to the point of preventing me to work correctly. After a bit of dusting out etc. I solved my problem by changing the 4GB memory module of my Lenovo X220 and putting a 2GB I had from another Lenovo. It worked. Good and bad. Good, it works, really bad, I cannot run anymore my virtual machine for my customers.

So, I had no choices but looking for RAM modules and I must say, I am not used to do it. From the documentation, I had to find the very special 1.5V CL=9 DDR3 1333MHz SODIMM and if possible from a good company like Crucial. On the website from Crucial, it offers the use of the low voltage 1.35V but on the website of Intel, the only supported voltage is 1.5V.

Anyway, I was lucky, on Amazon it was possible to order two 8GB 1.5V CL=9 DDR3 1333MHz SODIMM from Crucial.

Happy days until opening the box, the delivery was composed of two 8GB 1600MHz 1.35V CL=11 SODIMMs. So, I tried my luck, just putting one SODIMM and restarting the laptop. It worked!

Basically every body is contradicting everybody, be it on a discussion forum or any other place. The reason is simple, new low voltage RAM modules can run both at 1.35V and 1.5V. For the CL or CAS latency it looks like the system can adapt itself. I have no idea how, but the Crucial website tells me that I can use some CL=11 modules.

Now, I went from 4GB to 16GB in my laptop for less than 100€, with my first computer having 1MB of RAM, I am always surprised by the cost of electronics.

For the reference, the 8GB modules I bought are the CT8G3S160BMCEU.

Of course the question now is: Was it really the memory module or a simple bad contact or something like that which was solved when opening/closing the case of the laptop? I do not have the time to answer, but I will be able to test in the future by putting the 4GB module in place of the 2GB in my old laptop.

VMWare and Your Keyboard Mapping

2017-02-01T13:00:00+01:00

To be really productive for my customers I have, over the years, adapted my work setup to my needs. One of the keys of my productivity is the use of wonderful TypeMatrix 2030 keyboard with the Dvorak layout.

The other key is the use of Emacs as text editor. One peculiarity of Emacs is the extensive use of the Ctrl key to run special commands. In a text editor you can save your work with Ctrl+S, copy with Ctrl+C and paste with Ctrl+V, in Emacs can do nearly everything with some Ctrl+key combinations. This requires an easy to access Ctrl key and on the TypeMatrix 2030 keyboard, this means swapping the left shift and the left control keys.

On a modern Linux system with systemd, remapping keys of a USB keyboard is pretty simple, you create a file which simply tell which key code will match which key for which device.

I created the file /etc/udev/hwdb.d/80-typematrix-key.hwdb with the content:

evdev:input:b*v1E54p2030*
  KEYBOARD_KEY_700e0=leftshift  # bind left control to left shift
  KEYBOARD_KEY_700e1=leftctrl   # bind left shift to left ctrl

and run the commands:

sudo udevadm hwdb --update
sudo udevadm trigger

now the two keys are swapped and the changes persist across reboots. If you need, you can learn more about scan codes and key codes. Everything work, I am happy and go back to work. This was just after an upgrade from Ubuntu 14.04 to Ubuntu 16.04 which went without a single glitch. Happy day.

The next day, I run an upgrade of VMWare Workstation as my old version (9) was not compatible with the current Ubuntu version. VMWare Workstation belongs to the software I am happy paying for. They make my work so nice. Because the last thing I want to happen as I work for my customers is the leak of data out of my system as most of the bits I move are covered with non disclosure agreements. This results in a secure by default approach with the rule that I isolate the work for each customer in a dedicated virtual machine.

So, I install the new Workstation, launch a virtual machine, click around and try to go back to work. End of the day for two days. My keyboard went totally fubar, enable to use the Ctrl keys anymore, both on the left and the right. Of course, as the day before I moved from xmodmap to the new udev/systemd approach, I first tried to figure out what was going on on this side of the system. Without any luck and success, I tried to get logs out of systemd and the system messages.

Then by luck, after an afternoon of frustration and a good night, the next day, I found the answer and the issue is coming from the Gnome handling of the capslock key remapping. The solution is extremely simple:

In Gnome Tweak Tool, set "Ctrl key position" to "Caps Lock as Ctrl" and "Caps Lock key behavior" to "Caps Lock is disabled".

Nothing more, nothing less. It works.

Bonne Année 2017

2017-01-31T09:52:00+01:00

Already a month in, in France this is the last day to wish everybody all the best for 2017. You can read my best wishes for 2017.

Christmas Holidays

2016-12-22T11:51:00+01:00

After a long year, with a lot done and probably even more to be done in 2017, we all need a good break for Christmas. The office will be closed from tomorrow the 23rd of December until the 3rd of January inclusive.

Phone calls will not be taken but I am on duty in case of emergency. Please contact me per email if you really need my help. Emergency email will be checked every day.

Normal work and answers to your questions will resume on the 4th of January 2017.

Enjoy your break and rest well before starting in the new prime year 2017!
Loïc

Windows 7, Update from SP1

2016-10-17T13:36:00+02:00

For the last 3 days, I have been living within Windows Update. I am changing my machine from a 32bit version to 64bit version of Windows 7. This is the professional version of Windows 7, nicely installed from an original DVD exactly the way Microsoft want us to do. It has been 3 days in the making, I cannot count the numbers of restarts and it is still to be completed.

If you really have to do this, if your DVD contains Windows 7 SP1, do not use Windows Update.

Go download the July 2016 update rollup for Windows 7 SP1 and then start the update using Windows Update. Note that this rollup update is a nearly 600MB download.

Why going through this pain? This is to be able to build the new Cheméo Studio and deliver it for both 32 and 64 platforms.

I Like Fortran

2016-09-27T18:34:00+02:00

Fortran is really a nice language when it comes to scientific computations. So nice that I have some difficulties when using a different one like Python.

If you do science with Python, you always end up with using numpy, a wonderful extension to work with arrays and matrices efficiently. But... there is a but... it is not statically typed and can create some funny results, like when you multiply a matrix by a vector with a dot product.

>>> import numpy as np
>>> A = np.asarray([[1.0, 2.0], [3.0, 4.0]])
>>> B = np.asarray([5.0, 6.0])
>>> np.dot(A, B)
array([ 17.,  39.])
>>> np.dot(B, A)
array([ 23.,  34.])

What is happening here is that when you run np.dot(A, B), as B is a simple array, it is replicated two times to match the size of A, so you effectively run: 5x1 + 6x2 and again 5x3 + 6x4.

When you run np.dot(B, A), you run in columns: 5x1 + 6x3 and 5x2 + 6x4.

Numpy is smart, but deep into your computational code, an order swap or an error in dimensions can result in really hard to track errors. Fortran will simply send you back to the drawing board telling you that the dimensions do not match and that you should work a bit more.

Simple Secure Backup for a Consultancy

2016-09-15T15:03:00+02:00

Nowadays, with the cloud, you have a large number of companies offering backup solutions for your computers. The only problem for most of these solutions is that they are in the cloud, this results in:

low recovery speed as you need to download again GB of data over a relatively slow connection;
dependency on a third party and companies are going bust;
possibly absence of a cloud offer matching your country regulations (location of the data) at a reasonable price.

If the cloud is not matching your requirements, here is a very simple solution based on external USB hard drive. The goal is not to be perfect but to be simple.

What you need is:

an external hard drive, for example the WD Elements 2TB drive;
a hardware token to store a long password, for example a Yubikey;
a labeling machine, like a Brother P-touch;
a safe at your bank;
the ability to compute a bit in your head.

Everything is done on a Debian based system using the Gnome desktop. Read the full article before starting to follow the instructions.

First you need to store a long static password on your Yubikey. You should print a copy of the password and put in the safe at your bank for backup purpose.

Then, simply plug your USB drive into your computer and open the disk utility. Using the disk utility, stop the drive and remove the existing partition.

Then create a new encrypted partition.

The passphrase is important: in my case, I have a simple to remember but hard to guess algorithm to generate the first part of the passphrase from the serial number of the hard drive. Think something in the line of using every second character but not the numbers and adding something in between. The second part is simply the long password from the Yubikey. Write down the algorithm on the paper with your backup of the long password and be sure to put the paper in the safe at your bank.

Now, you have an encrypted external hard drive. Because of the combination of an algorithm to compute the first part of the passphrase and the Yubikey (or any hardware token able to generate a password), it is easy for you to unlock the drive.

Now, every 2 to 4 weeks, just run a backup script to copy the full content of your laptop on the hard drive. The best is to use 2 hard drives and rotate them, keeping one in the safe at your bank. Once a drive is full, just buy a new one, never recycle the drives.

Do not forget to label your drives with your labeling machine. This is a picture of my first drive, it was full after 5 backups, as I just do a backup once a month, the costs stay at less than €200/year, which is fine.

In my case, this is combined with an automatic encrypted backup on a remote server performed using rsync. The automatic backup does not keep historical data like the backup on the USB drives but is automatic.

The backup script in use is this one. Note that I have special case to backup VMWare virtual machines which are not stored in the user home directory. This is for performance reason, the /home/vmware directory is not encrypted but the home directory is.

#!/bin/sh
set -e
LOGIN=$USERNAME
YEAR=`date +"%Y"`
DATE=`date +"%Y.%m.%d"`
DRIVE=$1
: ${DRIVE:?"You need to provide the path to the drive."}

echo Start backup to $DRIVE.

if [ ! -d "$DRIVE" ]; then
    echo "Missing backup folder: $DRIVE."
    exit 1
fi

echo Make base directories in $DRIVE/$DATE.

mkdir -p $DRIVE/$DATE
mkdir -p $DRIVE/$DATE/$LOGIN
mkdir -p $DRIVE/$DATE/vmware

echo Get software selection.

apt-mark showauto > /home/$LOGIN/Library/installed-pkgs-auto.lst
apt-mark showmanual > /home/$LOGIN/Library/installed-pkgs-manual.lst

echo Backup /home/$LOGIN to $DRIVE/$DATE/$LOGIN.
sudo /usr/bin/time -f "\t%E real,\t%U user,\t%S sys" /usr/bin/rsync -aH --stats \
    /home/$LOGIN/ $DRIVE/$DATE/$LOGIN/

echo Backup /home/vmware to $DRIVE/$DATE/vmware.
sudo /usr/bin/time -f "\t%E real,\t%U user,\t%S sys" /usr/bin/rsync -aH --stats \
    /home/vmware/ $DRIVE/$DATE/vmware/

## To restore:
## sudo apt-mark auto $(cat installed-pkgs-auto.lst)
## sudo apt-mark manual $(cat installed-pkgs-manual.lst)

echo Done

At the end, a simple and worry free strategy.

Cheméo Studio Intuitive Prediction Check

2016-08-18T15:30:00+02:00

The work on the desktop component of Cheméo, Cheméo Studio, is continuing. The automatic check of the property predictions based on similar molecules with both experimental and predicted values is working wonderfully. The parity plots are now interactive and three times faster to draw. This means, you can easily analyse the prediction quality of 4 properties in real time.

The added benefit of the new graph system is that the resulting executable is smaller. Yes, smaller and faster, this is often the case with computer code, the less you do, the faster you are.

With the current state of development, I expect the first release of Cheméo Studio in September.

Bayer and the Porsche Effect

2016-05-30T10:00:00+02:00

Now, we know why Bayer decided to sell the material science part of the company to create Covestro: they simply wanted to buy Monsanto.

The cat is out of the bag and billion dollars question is: "Will it work?". For Bayer, this would definitely be a sucess if it works, but they are going to face two issues:

the regulatory issue, Bayer, Du Pont, Monsanto and Syngenta are the 4 big players, if Bayer buys Monsanto, the anti-trust regulations can kick-in and prevent them to buy Monsanto. This is not sure because a lot of small companies are providing fertilizers, crops, pesticides, fungicides and herbicides, but the global market share of the resulting entity would be 25% for the chemical part and 33% for the seed part. This at the global level, but it could be monopolistic in some regions of the world.
the Porsche effect. Porsche tried to buy Volkswagen in an under-cover all cash action, the resulting high-debt lead Volkswagen to buy-back Porsche for cheap. Bayer has a lot of debts, this could make it a nice target for yet another take-over.

The last point is the image. Monsanto in Germany is basically the devil but Bayer is a very well respected company. The green party in Germany is relatively strong, but stronger is the general green way of thinking of the german population. How the new Bayer will be able to overcome this image issue? Will Bayer be able to keep attracting great engineers? The oil an gas industry has difficulties in recruiting in Germany also because of the bad press it has, will this have an effect on Bayer too?

From a personal point of view, I do not think this will create better value for the people on the long run. It will definitely create better value for the shareholders, but for the farmers and the customers like you and me taking care of our garden, this is unlikely to have any real positive effects. Why? Because even if Monsanto and Bayer Crop Science are criticized a lot for their pesticides, they are already doing great research and development and the merging will not bring more innovation to these already highly innovative and active companies.

Cheméo Studio Explore Molecules

2016-04-27T15:30:00+02:00

The work on the desktop component of Cheméo, Cheméo Studio, is nicely going ahead. The robustness and ease of use is really one step ahead of what is available at the moment. Maybe because the motto of Cheméo Studio is: "Do one thing and do it well".

To illustrate how it works, here are two small videos:

The interface is simple and fast. It requires a bit of memory because of the drawing of the similar molecules are precomputed and the database has more than 20,000 molecules, but the speed is there. If you want to have an early access to the software, just drop me an email.

Update: You can now easily load a SMILES, here I copy/pasted the DDT SMILES from the PPDB databank. To load a SMILES, simply click on the File menu and select Load SMILES.

Easter Holidays

2016-03-23T14:30:00+01:00

Short break for Easter, from Friday the 25th of March to the 3rd of April, the phone calls will not be taken. In case of emergency, you can simply send us an email at the normaly email address. Emergency will be checked every day.

Normal work and answers to your questions will resume on the 4th of April 2016.

Enjoy your Easter holidays!

Easter Holidays

2016-03-23T14:30:00+01:00

Normal work and answers to your questions will resume on the 4th of April 2016.

Enjoy your Easter holidays!

Cheméo Studio is Coming

2016-03-17T22:00:00+01:00

If Cheméo usage of the chemical property prediction tool is growing, some users would like to run the predictions on their own desktop. This is really important for confidentiality reasons.

I am happy to announce that Cheméo Studio is coming!

The 2D molecular drawing tool is extremely easy to use and the prediction methods will include the good and well known Joback method together with the Constantinou-Gani and Marrero-Gani methods.

With 15 years of experience developing and maintaining the ICAS software at CAPEC together with countless hours discussing their needs with engineers from the most innovative companies, I can tell you that this software will simply be the best in its class.

If you are interested in testing the software before general availability, just let me know.

Update: A long requested feature is now available, the instant similarity search in the database while drawing the molecule. The similarity search takes about 70ms to run. As the "instant" feeling for desktop application is 100ms, it means I do not even need to show a kind of "loading similar compounds" overlay.

Keeping Cheméo Fast

2016-03-02T10:00:00+01:00

With the time, the usage of Cheméo is growing, this is nice, but for the users to be happy, the speed of the system must stay the same or improve.

Early this year I have rewritten the backend storage of the data and the indexing system. The results speak from themselves.

On the graph, you can see the time it takes for the software to render a page for you, the end user. The page can either be a compound page like the chemical properties of Propanol but also the corresponding drawing, Excel or PDF export. The percent are in fact the percentiles, that is, if you follow the 95% line, you know that 95% of the pages are generated in less than this time.

What is very impressive is that even the 99% is nearly always below 100ms. 100ms is the "instant" reaction time. That is, it is the time for the user to have the feeling the system is reacting instantaneously (See study).

Of course, this is just at the "server" level, the generated page must then be sent over the wire to the end user. This can take way more time, especially if you are looking for chemical properties from India or South Africa. But the good thing is that the pages are also pretty small in size, with the chemical properties but no funny effects and scripts and thus can be correctly delivered even if you have a low quality bandwidth like in Kazakhstan or French Polynesia!

The last interesting point of the rewrite was that it can now be installed easily within the Intranet of a company. But this is more a business point of view.

Cheméo Crawler is Crawling Again

2016-02-18T10:00:00+01:00

Good feeling, the Cheméo crawler is now crawling again the web. The crawls are still performed in batchs, that is, the crawler is only launched every few months, not like Google which is accessing the websites every day.

This time, the crawler has been improved to discard non interesting pages before crawling and rewriting the URL to avoid duplicating the crawl. For example the two URLs:

http://my-website.com/index.php?foo=bar&bing=bong
http://my-website.com/index.php?bing=bong&foo=bar

are the same, because the query string bing=bong&foo=bar passes two keys bing and foo with values bong and bar. By rewriting the URLs before the crawl, the system avoids duplicate crawling. Both the crawler and the targeted website are now doing less work.

The crawler is alse using bloom filters to efficiently discard already crawled links and bad links and the best part of it is the storage of the retrieved data in a WARC file.

A WARC file, the Web ARChive file format, is a standard to preserve the content published online. It is a well though ISO standard with a pretty large number of tools available to manipulate them. The advantage for the crawler is that many versions of the same page can be stored, it is compressed and can be efficiently manipulated after the creation. This way, the parsing of the pages is performed in total independence of the crawling.

The current crawling session is expected to last approximately a month.

Update, 23rd of February: nearly 200,000 URLs crawled in 5 days, with 1.2 millions to crawl, the one month crawl is on track.

Christmas Holidays

2015-12-21T14:23:00+01:00

Winter is here but the weather feels way more like spring. We have cherry blossom in the city just nearby, this is worrying.

Anyway, from the 23rd to the 4th of January, the office will run in emergency mode only. The phone calls will not be taken from the 23rd to the 27th of December inclusive and from the 31st to 3rd inclusive. In case of emergency, you can simply send us an email at the normaly email address. Emergency will be checked every day.

Normal work and answers to your questions will resume on the 4th of January 2016.

Enjoy your Christmas and new year break!

Upgrade of our Internet Link

2015-12-03T11:00:00+01:00

Tomorrow the 4th of December, Telekom is supposed to switch us on the new 100Mbps connection powered with the so called vectoring technology. I am not sure how well the transition is going to work. In April we moved from a stable ISDN system to a full IP connection, so normally, it should be just a switch, restart the router and enjoy the new speed, but who knows?

In short: tomorrow, the 4th of December, we me have no reliable internet connection.

Update: From the 4th of December 13h45 to right now, the 7th of December 10h00, we had no internet connection and because the phone is now running over the internet connection, no phone. Nearly 3 days. Not good. We now have a 65Mbps downlink and a 35Mbps uplink.

Cheméo API for Research

2015-11-20T14:00:00+01:00

Since the release of the Cheméo API the API starts to be used.

This was with a great pleasure that, few weeks ago, I received an email complaining about a bug in the API. It is not everyday that one is happy about a bug report!

The report was from Daniel O'Neil, a researcher at the Georgia Institute of Technology. He is using the Cheméo API to experiment with machine learning algorithms to predict properties of compounds. I was really pleased with this use of the API, as I was doing my PhD, it was always a problem to access data in an easy to use format. This is great to see that my work is making the life of researchers easier.

You too can use the Cheméo API, it is available for free.

A New Molecular Drawing Tool

2015-09-28T10:00:00+02:00

For the prediction of chemical properties with Cheméo I am using an open-source tool named Ketcher. It is working pretty well but has two inconvenients:

it is big and can do a lot more than what we need now and in the future;
it has a license preventing us to integrate it into a standalone close source software (AGPL).

The last point, the license, is not really a problem as you can get a special licensing agreement from the authors of Ketcher. The second point can only be solved by shopping around for something different.

After a while, I saddly found nothing matching my requirements, this is why I started a new system. The picture above is a screenshot of a drawing with this new tool. The tool is simpler, easy to maintain and extend and plays well with the libraries I am using for the rest of Cheméo's interface (jQuery).

The current software in testing weights 840kB and can be compacted in two files (CSS + Javascript), Ketcher weights 3.1MB in 170 files (a call to the prediction page issues 100 requests to load everything).

I hope to be able to switch the component to allow you to draw faster with a simpler interface during the month of October.

Update: Now it works pretty well, of course the mandatory drawing of the Trinitrotoluene with the new tool!

Cheméo API Available

2015-09-15T22:00:00+02:00

The first version of the Cheméo API is available. This is a REST API and it is very easy to use it. You basically issue a specially crafted GET request against the Cheméo API and you get a JSON formatted message as response.

At the moment, the available tools are:

conversion of from a name, SMILES, InChI, InChI key and Cheméo ID to another format.
generation of the structure of molecule from its name using the IUPAC rules.
property prediction using the Joback, Abraham and McGowan, and, Crippen methods.

The access to the API is free and the Cheméo API is fully documented.

Update: You can now run a search against the database, for example, using the command line on a Linux/OSX system:

curl -X GET "https://www.chemeo.com/api/v1/search?q=methane"

Learn more about the search API.

More than 350,000 Pure Compound Properties

2015-09-02T13:23:00+02:00

At the moment, this is in fact 357,266 pure compound properties that are evaluated. You can expect to have them in Cheméo pretty soon. This is really great news and this part of a work to ingest about 1.5 millions pure and mixture property datapoints in the database.

These data are covering around 2000 compounds/substances. The work is complicated by the lack of clear information about each compound, when the data are covering Water, this is pretty easy to figure out which corresponding compound si matching the database. But when we handle tetrabutylammonium dodecyl sulfate you need a bit of manual work to be sure the match is correct.

Bye Bye Céondo Ltd, Hallo Céondo GmbH

2015-09-01T15:23:00+02:00

Short but important notice. Céondo Ltd is going to be struck off from the register at the Companies House within a couple of weeks. This will not surprise anybody, but legally this is important.

All the customers with some contracts under the Limited company have been informed, but maybe I missed somebody.

Céondo GmbH will soon be the only existing structure as the limited company is already a defacto dormant company.

Differences between C, C# and Fortran

2015-06-19T15:23:00+02:00

Some days ago, I enjoyed a very nice discussion with a couple of persons, the majority of them not having programming as their main center of interest. But, we ended up comparing C, C# and Fortran.

It was a lively discussion, so on the fly I tried to find a way to understand the main differences for somebody having no ideas about programming. I focused the difference on the memory management part. Here is the way I tried to express the differences.

Imagine your program is doing some work like you on a desk. You need paper and pen. The paper is the memory used by the program.

In Fortran, before you start to do the work, you grab sheets of paper, you mark them to tell when and for what part of the work you are going to use them. Once done, you start to work. When you start a part, the sytem will give you the paper you need (because you marked them) and will remove them at the end. If you need more paper at some point than what you decided at the start, sorry you cannot.

In C, you do not anything at the start. When you start to work on a part, you pick the paper you want. At the end of each part, it is up to you to clean your desk or keep some paper for the next part. But if you forget a sheet of paper during a part, you can start to accumulate trash on your desk.

In C#, it is a bit like C, you do nothing particular, but when you need some paper, you ask your assistant, your assistant do not know your needs, so he will give you a bit more or not enough, so you may need to ask more or you may have a bit too much on the table. What is nice is that your assistant is learning, it improves a bit with time. At the end of a part, the assistant will also carefully inspect your work and remove what he considers are the sheet of papers you do not need anymore. This is nice, but it costs a bit of time.

At the end, in Fortran you need to define everything before, but the results are a very fast system, in C you can do some nice optimisations but you need to manage a lot and this is error prone and in C# you enjoy the work of your assistant at the cost of some inefficiencies here and there.

ESCAPE25/PSE2015 Conference

2015-06-10T14:30:00+02:00

It was with a great pleasure that I gave a talk on the uncertainties in the prediction of chemical/physical properties using group contribution methods at the combined PSE2015/ESCAPE25 conference. The conference was extremely well organized by Prof. Gani and his team.

The keynote was well received and generated a lot of interest by the people in the room. You can already download the slides. The main takeaway is that at the moment the prediction interval for the group contribution methods (QSPR) is not fully reliable.

A paper going more in depth is in writing at the moment, so you can expect a publication within the next 3 months. The work in progress is bringing solutions to this issue.

Send Us Encrypted Emails

2015-05-07T14:00:00+02:00

This is technical piece of information for the security and privacy conscious people communicating with us.

The GPG key associated with the email address loic@ceondo.com has been renewed. You can download the public key from this server or get it from your keyserver. The key ID is F89B0636.

This is part of our ongoing work to secure the communication with our customers. The previous work was to only allow access to the company website over SSL.

Small Tip if Using Enigmail on Ubuntu Linux

If you are getting the error OpenPGP warning, bad passphrase but everything is working correctly using the command line. You have most likely the wrong pinentry link, going to the qt4 version instead of the gtk-2 version. You can easily change the link, just run as root or with sudo:

# update-alternatives --config pinentry
There are 2 choices for the alternative pinentry (providing /usr/bin/pinentry).

  Selection    Path                     Priority   Status
------------------------------------------------------------
* 0            /usr/bin/pinentry-qt4     95        auto mode
  1            /usr/bin/pinentry-gtk-2   85        manual mode
  2            /usr/bin/pinentry-qt4     95        manual mode

Press enter to keep the current choice[*], or type selection number: 1
update-alternatives: using /usr/bin/pinentry-gtk-2 to provide
   /usr/bin/pinentry (pinentry) in manual mode

Experimental Data when Predicting Properties

2015-04-28T13:00:00+02:00

When you are predicting properties on Cheméo, the system is automatically looking in the database for a matching molecule. If a molecule is available, you directly get the known experimental property ranges. For example, for the 2-methyl-1-Butene you get:

You can experiment yourself starting with the predictions of 1-Pentanol.

The next step will be to provide an easy access to the predictions for similar molecules.

Predict logP and McGowan Volume

2015-04-21T17:30:00+02:00

It is now possible to predict the Octanol/Water partition coefficient (logP) using the Crippen method and the McGowan volume on Cheméo.

For example, you can run the predictions for Aspirin.

The planned improvements for the next weeks are the automatic pull of the experimental data for the molecule being drawn and a list of similar molecules. The list of similar molecules will be relatively easy, albeit the need to control the search timeout to keep instant answers at the possible cost of not updating the list. We will see in practice the response time effec to still get a desktop feeling when using the application.

10 Second Power Cut in a Refinery

2015-04-21T13:29:00+02:00

Just around the corner of Céondo offices, we have a big refinery, in fact, this one of the biggest in Germany, BP Gelsenkirchen.

End of March, the sky at night was of a beautiful red/orange color. They suffered a 10 second power cut, this resulted in all the processes stopping and putting themselves in security and wonderful flaring. Ok, this is maybe not that great for the environment, but this makes a nice show.

Normally the refinery, which by the way has a power requirement equivalent of 400,000 households (back of the envelope calculations, 400,000 x 2,500 kWh/year = 1GWh/year) gets its power from 2 independent networks. A failure of a power substation of one of the network few kilometers away from the refinery resulted in a cascade stopping the 2 networks.

After this 10 seconds interruption, it took more than a week to bring back the refinery to full production.

Easter Holidays

2015-04-02T15:29:00+02:00

Spring is here and Easter too. The office will be closed from the 7th to the 10th of April. It will also be exceptionally closed Monday the 13th of April.

From the 3rd to the 12th of April, we will not be available over the phone but per email and it will take up to 72h for you to get an answer to your non urgent questions. Urgent issues will be taken care of within 24h as usual.

Exceptionally, Monday, the 13th of April, nothing will be done. Ok, maybe we will be able to work, but on Monday our office is getting an upgrade of the phone and internet connection and I must say, I have no ideas if it will work or not! This is a partial upgrade before getting a 100Mbps connection later this year.

So, enjoy a nice Easter break and do not break everything will we sleep a bit.

Update 2015.04.03: Nothing left in my inbox! This is nice before a break. I will check my emails on a regular basis to avoid an overflow when coming back.

Update 2015.04.15: Finally back with an internet connection. If our connection with Telekom is changed from Universal ISDN to IP based and you get the following errors with your ADSL router (in my case a Fritz!Box 7270):

Anmeldung beim Internetanbieter ist fehlgeschlagen. 0001 STPR2AU
0000000000000 Zugriff verweigert.

But you have a stable DSL connection, just request the technical support to send you a complete new set of connection information (Zugangsdaten) per email. It will solve the problem immediately.

Easter Holidays

2015-04-02T15:29:00+02:00

Spring is here and Easter too. The office will be closed from the 7th to the 10th of April. It will also be exceptionally closed Monday the 13th of April.

So, enjoy a nice Easter break and do not break everything will we sleep a bit.

Update 2015.04.03: Nothing left in my inbox! This is nice before a break. I will check my emails on a regular basis to avoid an overflow when coming back.

Anmeldung beim Internetanbieter ist fehlgeschlagen. 0001 STPR2AU
0000000000000 Zugriff verweigert.

But you have a stable DSL connection, just request the technical support to send you a complete new set of connection information (Zugangsdaten) per email. It will solve the problem immediately.

More Molecules in Cheméo

2015-03-27T10:15:00+01:00

The Cheméo crawler is working, working, working. It is a slow crawler, pulling about a page out of the Internet every 3 seconds, respecting all the robots.txt rules not to saturate the indexed databases. But when you do something slowly, regularily, without interuptions over a long period of time, it works.

Cheméo is now indexing more than 70,000 molecules and this will not stop.

You can take a look at the latest index statistics about the chemical properties.

logP, Octanol/Water Partition Coefficient

2015-03-16T10:15:00+01:00

The logP or Octanol/Water partition coefficient is now available for nearly all the molecules on Cheméo.

The logP is predicted using the Wildman and Crippen method: "Prediction of Physicochemical Parameters by Atomic Contributions". You can download the full text as pdf, if this is not available anymore, just ask me and I will send it to you.

For the moment, this method is not available in the online prediction too but it will be added in the future.

Here is a search for molecules with a logP between 2.3 and 2.7.

Cheméo is Fast

2015-02-24T11:15:00+01:00

The main problem of the chemical database is that they are slow. When you start to think abouting using a chemical database, you usually start to think at the same time about pouring yourself a new cup of coffee or tee.

When developing Cheméo, the rule was simple: it must be fast, very fast. The end result can be seen on the graph, Google takes on average of 62ms to download a page. This is faster than 99% of the websites. To achieve this, I even wrote assembly language to have the fastest similarity search with custom merge and sort algorithms to bring the results together.

So, if you need a fast access to a chemical database, just try Cheméo!

Cheméo is Back Online

2015-02-20T21:15:00+01:00

It took a month longer than planned, but I must say, I am really happy to have Cheméo back online! I am not totally sure about the stability of the new system as I am experimenting a bit with a new Docker based deployment system. For the moment, it runs smoothly.

A really nice thing for you is that you can run physical/chemical property predictions using the Joback Method directly in the browser. This is done in real time.

You can also find similar molecules to a target. It is easy because you simply provide the target as a SMILES. If you can't remember the SMILES, you can draw the molecule with an easy to use molecular drawing tool.

The next things to come are:

adding a bit of copyrights and links to the software used;
opening the user area to manage your list of molecules;
opening the access to the regression tools.

But all in all, I am very happy.

Update: Working like a charm, the site is fast and very responsive. I can now start to deploy the more advanced features.

Property Plots on Cheméo

2015-02-18T14:20:00+01:00

As you can see on the attached screenshot the new Cheméo will include property plot of the temperature and pressure dependent properties.

What is really nice is that you also get the incertainty on the value on the plot. In this example, it is easy to spot that the ideal heat capacity of the Propanol was reliably measured.

Do not worry about the performance impact of such plots, they are precomputed to keep a very responsive website. If you are still wondering, the new Cheméo should be open for anonymous use within the next few days. At the moment, I am mostly cleaning the style and ensuring it looks nice. All the functionalities are implemented and the database is ready.

Update on Cheméo

2015-02-05T14:20:00+01:00

If you are wondering, here is a quick log about the relaunch of Cheméo. You can get the log on the homepage of Cheméo but some of you are following the news here. So, here you have:

30 Jan: Since the 28, the new crawler is performing pretty well. We will soon have a fresh dataset to start againg Cheméo.

2 Feb: I guess we have now 90% of the data, the last 10% will most likely require as long as the first 90%. Not to wait too long, the parsing will start tonight or tomorrow and the remaining data will be integrated later.

3 Feb: The parsing is going well, the next step is merging the data to build for each compound a record. From this record analysis and computations are performed. The critical step will be the refreshing of the record with the data from the crawler.

4 Feb: Crawling, parsing and merging are operating very well together. The pipeline is running smoothly. Next step is performing the calculations and generating the indexing data. This is a bit harder. I will most likely start it tomorrow.

12 Feb: Some quick stats, the parsing of the crawled data takes about 5h, the merging of all the documents referring to a single compound into a master document takes 1h something and the extension and indexing of each master document to generate the ready for production document is not yet completed. It has a lot of control cases and I am clearing them as they come.

13 Feb: The first batch of ready for production document completed... Now the fun part is coming, insertion in the index and relauch! Update: first insert was ok, I will start the validation on the test server soon.

New Céondo Website

2015-01-12T14:20:00+01:00

7 years, it has been 7 years since I created this website and it did not change for the last 7 years. This was really time to get a redesign.

Refreshing the identity of the company is not an easy task, because, if the previous design was here for 7 years, I can expect the new design to also last 7 years. This resulted in a couple of simple questions to answer:

How would I design a website for 7 years?
What are the current trends which are going to flourish in the next years?
What is the simplest system to ensure worry free operation for another 7 years?

A New Design and Identity

The new design is basically a simpler version of the old design, the main changes are on the motto of the company. After 7 years, I know what I do: Fluid Phase Equilibria, Chemical Properties & Databases and as such, this motto is everywhere on the website. You can see it at the bottom.

This results in a simplification of the logo, I no longer have the Software for the Scientific World line at the bottom. I took the opportunity to rotate the star of the logo to have the top edge totally horizontal, this has a better rendering at small size as it is possible to be pixel perfect.

Basically, a bit simpler but still the same.

A New System to Design the Website

Mobile first, if you are building websites now, the trend is definitely to provide a mobile optimized version of your website. I must say, for the moment, mobile browsing of ceondo.com is not my priority, but it should be possible to provide it in the future without problems.

This why I am using SASS. This is basically a layer on top of the traditional CSS and it allows me to have cleaner style sheets and take into accounts the different browser versions easily. A simple example is that you can have variables in your CSS files, for example, for the color of a border and you can reuse the variable everywhere.

A New System to Publish the Website

No software on the server, just plain HTML files build from a canonical source of content using the Pelican software. Since my first website in 1994, plain HTML pages work very well and never break. The canonical source of content is a simple stack of utf-8 encoded simple text files looking like:

Title: New Céondo Website
Tags: News, Design
Date: 2015-01-12 14:20
Image: /images/2015/v1.ceondo.com.header.png
Summary: After 7 years, it was ... the company website.

7 years, it has been 7 years ... this website and it did
not change for the ... This was really time to get a
redesign.

Etc...

Very simple to manage and generate. It was very easy to import the old website into this format.

The added benefit is that the website is extremely secure. The website is also now delivered over an encrypted connection (SSL).

Still Some Work

I still need to do a couple of things, this includes:

automatically generate the work reports from my time tracker for my customers. They of course should be protected with a login/password.
get a protected file download area for each customer.
some old blog posts have lost some pictures.
improve the home and services pages, I am not totally satisfied with them.
pushing new content on a regular basis.
get the new years card script to work again (not yet sure how I will do it).

The styling and the content will also need to be improved with the time based on user feedback and my personal taste.

Emails Up and Down

2014-08-25T07:59:26+02:00

Just for your information, my emails have been bouncing a bit in the past few weeks. I am not sure why as I haven't changed my email provider nor my email settings. If you haven't received an answer for something important, I may have simply not received your email. Be kind enough to send it again. Thank you! Loïc

Back from being away with some work to do

2013-08-29T09:25:30+02:00

Backup, backup, backup. During my stay away, one of the drives of one of the nodes in my cluster failed. The problem is that it was an half backed failure, with sporadic failures leading to corruption. So, I lost my ability to communicate by emails during the last 3 weeks. Not fun because from my point of view I was able to send the emails, but most of the emails ended up in a computer black hole.

If you communicated with me during the last 3 weeks by email, please accept my sincere apologizes. If you do not get an answer to your inquiry within about 8 days, please send your email again, my emails are hosted by an independent provider, but the DNS server is hosted on the cluster, so the hard drive failure may have affected DNS resolution and thus email transfer.

Everything should be back to normal by Friday the 6th of September.

End of Life of Indefero Hosting

2013-07-05T12:56:39+02:00

Reminder, the end of life of Indefero hosting was reached on the first of July 2013. You had 1 year to adapt, so I hope you took the time to effectively adapt. The hosting will be effectively shut down the 15th of July 2013. If you need a PostgreSQL backup of your forge data on top of the JSON backup you get from your account, please contact me as soon as possible.

Indefero downtime, routing issue

2013-05-15T11:53:27+02:00

Wed May 15 11:53:09 UTC 2013: Hello, we have a routing issue between the frontend and the backend database of Indefero. The database is safe, nothing to worry, this is just that the frontend cannot connect to the database server like if suddenly a firewall was cutting the connection. We are investigating.

Wed May 15 13:54:26 CEST 2013: Our provider is having an issue, we keep an eye on it.

Wed May 15 13:55:39 CEST 2013: Ping between the machines went from 5 ms to back under 1 ms, it looks like our provider is doing something.

Wed May 15 14:02:45 CEST 2013: Our provider changed some configuration of their routers and they are now observing the situation, Indefero is back online.

Reminder: Indefero Hosting End of Life June 30, 2013

2013-05-06T08:04:13+02:00

Reminder: The Indefero hosting will stop June 30 this year, yes, 2013. You were informed about a year ago, so I hope you had time to migrate.

If you are a Subversion user, Assembla can import your dump of your repositories. The daily Subversion dumps are available in your account area. They also offer Git hosting. If you are using Git only, you have 1000's of offers.

Of course you can install your own version of Indefero and go ahead with a self hosted solution as Indefero is a free software.

Update of the PostgreSQL server for Indefero

2013-04-11T07:27:08+02:00

Thu Apr 11 07:27:29 UTC 2013: There is a pretty serious update of PostgreSQL to be done for security reasons. Indefero will be done for a couple of minutes the time to update the server. I am sorry this is an unplanned update.

Thu Apr 11 07:30:54 UTC 2013: The slave has been updated and is correctly picking up the updates from master, now going to update the master. From the slave update sequence you can expect about 2 minutes downtime of Indefero.

Thu Apr 11 07:33:36 UTC 2013: The master has been updated, the effective downtime was less than 20 seconds.

Electrical failure, Indefero down

2013-03-06T16:23:25+01:00

Wed Mar 6 16:21:45 UTC 2013, we have an electrical issue, for the moment Indefero is down, it will come up as soon as the electrical issue is resolved. More details will soon be available from our provider.

Indefero down at the moment

2013-01-21T14:23:26+01:00

Mon Jan 21 14:22:30 UTC 2013 A large array of servers are down in our datacenter. Saddly, the main database server of Indefero is down too. The backup server is running ok and is up-to-date. We are expecting feedback from OVH to assess if we need to switch over the slave or if we can expect the main server to come back fast enough.

Mon Jan 21 14:24:25 UTC 2013 It was fast, power supply issue for RBX4. Still waiting for more information.

Mon Jan 21 14:35:20 UTC 2013 Everything is up again, sorry for the inconvenience.

Servers Down at the Moment

2012-10-15T09:22:48+02:00

Our provider has some issues. Not so fun, the servers are down and I hope they will come up again as soon as possible. Basically, we have a split on the internal network, some servers can talk to each other, some cannot. Very annoying because it makes split brain situation. All the data are safe, but we have to wait until this is solved (and they are doing it manually).

Update: Everything up again, down-time of approximatively 30 minutes, I hope it was at the time of your coffee break.

Céondo Ltd, 5 Years of Activity

2012-10-02T09:21:08+02:00

5 years, the time is flying. In May 2007, I created Céondo Ltd, in September 2007, active trading started and in August 2008, I started to pay myself a regular salary. Wonderful time and even better time to come, after 5 years of working hand in hand with the industry, I finally know where I can make a change, where I can really add value and make the work of the chemical and process engineers easier, safer and faster.

Many thanks to all my customers up to now and of course, all the people I have been in contact with during these 5 years, you gave me a lot and I hope you have the feeling I gave you a lot too. The next 5 years are going to be fun!

CAPE-OPEN Meeting in Lyon

2012-09-24T11:57:25+02:00

I happily participated at the CAPE-OPEN European Conference in Lyon. I sadly missed the special interest group meetings at the end of the conference as I kind of skipped them in my planning (I though one had to be part of the SIG to participate but they were open to everybody).

CAPE-OPEN is a great set of standardized interfaces (API) — mostly COM but also CORBA in theory — to allow a unit operation model or a thermodynamic model to be used in nowadays a large array of simulation software. The API is slowly growing to cover nearly all the needs in the process and chemical engineering area.

Some companies have shown how a single implementation of a model can then be used in several simulation software all used within the same company. In large chemical companies, for historical reasons and to avoid a dependency on a single vendor, they tend to run several simulation software to do basically the same thing but not exactly.

In some other cases, I am a bit more sceptical, for example, if you have only one simulation tool in house and the tool is providing a clear and simple interface (for example the Fortran UAS in PRO/II) using CAPE-OPEN would require both mastering Fortran and C++ to implement a COM wrapper. This may not be the best choice depending on the in-house know-how.

As for any technology, you need to have a pragmatic approach but seeing an open standard used every day a bit more in the industry is always positive news. I am looking forward to do a bit more in the field while keeping a pragmatic approach, sometimes simple good old Fortran is better.

Indefero Hosting Will Keep Going

2012-09-13T10:40:59+02:00

Very good news, after a lot of discussion (because I want to be sure of the quality of the offer) the Indefero hosting offer will continue as it will be taken over by a small companies already used to work with hosting data and user information under strict security rules. The transition will be totally transparent for the current customers and users. I will keep you informed.

Indefero Hosting Stopping Dec 2012 to June 2013

2012-08-02T09:56:32+02:00

Front matters: This is the email I sent to all the current users of the hosting offer.

Dear Indefero Users and Customers,

today is not an easy email I am sending you, this email is to announce you the wind down of the Indefero hosting platform. The Indefero hosting will be stopped at the following dates:

free hosting, 31th of December 2012 (in 5 months);
paid hosting, 30th of June 2013 (in 11 months). If your renewal date ends before the date, you will get free hosting until the end.

To help you in the changes, the period of transition is as long as possible. Now that you know the meat of the subject, let me provide you with the whys, how and details. But first, stopping a service is not an easy task, it is especially hard because you, as customers, trusted me to provide long term high quality service and by stopping the service I am breaking this trust. For me, it is also hard because it means that I failed to correctly predict the future.

Thanks a lot for the trust you had in using Indefero and please accept my sincere apologizes for not providing you continued services for another couple of years.

If you paid a renewal or a new forge in the past 45 days, I can issue you a full refund. In this case I will ask you to migrate out before the end of the year. 45 days is the limit of the banking system.

How to Save your Data

Simple, login here:

https://www.indefero.net/account/

and download the backups (down the page). You get everything related to your forge and the data are compatible with Indefero, that is, you can install Indefero on your server and import the data.

How to Make a Successful Migration

https://www.indefero.net/account/

Click on the "configure your forge" link:

https://www.indefero.net/account/bp/

and update the personal domain to use a domain you fully control. If you are working for the foobar.com company, put something like code.foobar.com and get a CNAME record in your DNS pointing code.foobar.com to yourforge.indefero.net. Then, start asking people to use code.foobar.com to access your forge. After a while, nobody will use the indefero.net address and you will have full control over your forge.

The next step is to setup your own Indefero instance and import the data from the hosted forge, then switch the DNS to point to your own Indefero instance.

The end result is a migration without downtime and without disturbing your end users.

Tools to Make a Successful Migration

In October, you will get ready to use Amazon EC2 images which will allow you to do nearly "One Click" migration of your data from the hosted platform to your own Indefero instance. With an EC2 micro instance, it will cost you about $15 per month to run your own Indefero instance. I will also work with the current providers of Debian packages to be sure you can easily setup and import your Indefero forge on a fresh Debian system.

Why Stopping Indefero Hosting?

Because of focus, when I started Céondo Ltd, I had not really a clear picture of where I wanted to go and how, now, I know and the key is "Science", that is, I will fully focus on scientific software and consulting. In the last months, I was able secure 2 to 3 years of consulting pipeline in science, this is a clear indication that this is the way to go, a specialization in an extremely technical area where the barrier to entry is very high.

How Will It Affect the Indefero Software?

Surprisingly, I expect it to be positively affected. The last year I have been slowing down my involvement in the software because changing the software would also mean for me, applying the changes to 3000+ forges on a system not designed in the first place to accomodate so many forges. I was afraid of the consequence of a bad upgrade at such scale.

The time used to manage the hosting will in part be redirected to improve the software and the migration tools will also be used in parallel to allow us to perform automated testing of Indefero. We will be able to start an EC2 micro instance, test and stop.

The Indefero community is very active with an increasing number of users and packages for nearly all the current Linux distributions, the current goal is to have the Indefero packages distributed officially by all the main Linux distributions to ensure long term support. You will soon get the distribution specific packaging scripts be part of the source code of Indefero. This is critical for the long term support of Indefero and will help testing on a larger scale.

Alternatives to Running your Own Forge

The code hosting space is crowded, so crowded that it is hard to recommend someone. First, the real question is:

how critical is your code? Can you accept to have it hosted by a third party? Of course you currently have it hosted by a third party right now, but it may be a good time to rethink this question. I think it is critical enough to have full control over it, this is why I gave full control with CNAME, backup and OSS dump compatibility to you when providing Indefero hosting. I could not provide something I could not use personally.
then of course, you need to define what you want in terms of functionalities, version management software (Git, Subversion, Mercurial, etc.) and the contractual constraints (hosting location within/outside the US, price, owned by a big/small company independent/partially owned by venture capitalists). This is not simple, I have seen an increase of the number of forge creations since GitHub took venture capital money on board. So, it looks like some of you do not like to be dependent on venture capitalist controlled companies. You have the time to think about it.

If I had only one service to recommend, I would recommend Pikacode under the lead of Benjamin Jordan. They have been hosting code repositories for a long time and have been active contributors to Indefero. I trust them and they are real system administrators, used to managed some of the biggest website in France, saturating Gbps of bandwith during big events. They know their job very well.

Here is the "inline" advertizing from Benjamin for you:

Pikacode.com offers Git and Mercurial repositories hosting. Formerly knows as Intuxication, thousands of repositories have been created by our users since 2008. Pikacode goals are simple : easy, sleek and fast code hosting. We offer you 90 days of free trial for unlimited private repositories and collaborators with the following voucher : HELLO-PIKA.

http://pikacode.com/

If some of you can recommend code hosting companies, just let me know. Note that Pikacode also offer free public repositories.

What is Next?

I will setup in September/October a migration website:

http://www.indefero.net/migration/

this will be your portal to have everything you need to get a successful migration without disturbing your users and losing your data. It will be updated with the latest information, tutorials, possible alternative offers — basically, everything to help you.

Again Thank You

At the end, I can only thank you for your trust and the bit of travel around the Sun we did together. I am proud of what was achieved with Indefero and I am honored you trusted me, I am also sure you will find a good way forward.

Best regards, loïc

Update of Indefero SSL Certificate

2012-07-30T06:44:59+02:00

The [Indefero|http://indefero.net] SSL certificate expires today, it will be renewed during the afternoon, normally the update is performed without downtime. If downtime, it should a couple of seconds the time to restart the web server.

Update 2012.07.30 11:30 UTC: The renewal procedure is on the way, it should be finished in a couple of hours to have the real update at the server level done just afterwards.

Update 2012.07.31 07:40 UTC: The certificates are now updated and valid for another year.

Creation of New Forges Suspended While Updating the Website

2012-06-25T12:12:50+02:00

Please accept our sincere apologizes but the creation of new Indefero forges will be suspended this week while the website will be updated. The website will be updated by the end of the week (30th of June 2012) but the exact dates when the website will be updated is not yet known. If you want to create a forge, please try doing it by Wednesday the 27th as the update will most likely take place Thursday/Friday.

Indefero Database Failover

2012-05-25T11:54:56+02:00

I am sorry for the unscheduled downtime this morning, Friday 25th of May around 8 am UTC. A scheduled kernel upgrade of the server went not as expected. The kernel upgrade did went correctly on the slave, including reboot and resync, but the master failed to come up again. For data safety reasons, we performed a backup of the slave before promoting it as master and switching the application to use the new master. This backup is what took a bit more time than expected and resulted in the large downtime.

What is next?

Need to promote a new slave and get the master to logship to it, this will again force a small downtime of the master database the time to pickup the new configuration.
Need to be a bit more proactive in announcing the issues. I setup a small list for the very active users, but a know central place to publish updates would be better.
Need a regular backup of the slave to not have to perform it under pressure.

I will keep you informed. Sorry for the annoyance, sometimes issues happen and this one took me by surprise.

Update: Reviewing the logs, the combination of a VM + Hardware node restart, including KVM upgrade is most likely the culprit.

They use Indefero: Elveos

2012-01-31T13:17:10+01:00

A small notice to inform you that Elveos.org is using Indefero:

Elveos.org is a crowdfunding website for open source software. You are a free software developer? Elveos gives you a way to get paid for your work. You are a free software user? Elveos let you fund the features you need.

It reminds me KickStarter but more open. It is very nice to see offers targeting the OSS community.

Firefox is gone, Chrome is here, who is next?

2012-01-31T09:44:47+01:00

Can you remember the old days? Life was simple, everything worth something was simple Netscape Navigator. Then Microsoft decided that the web was important and they took over the world and took a nap. Reborn from the ashes of Netscape, Mozilla brought us Firefox. But then, Google decided that the web was too important not to have their own browser, here we are, Chrome is the new leader.

Chrome: 50.92%
Firefox/Geco: 22.60%
Safari: 17.54%
Android: 6.12%
Opera: 1.64%
Internet Explorer, less than 1%...

Click on the image for a larger version. These are the results of a nearly 100% traffic burst from Hacker News this WE. They are the trend setters as early adopters. They are going to jump on the next Chrome, Firefox, IE, Netscape Navigator when it will come. But here, for the first time I wonder. Who could be the next? Can it be that we reached the goal of the Mozilla Fondation of a competitive field, where several players are taking the lead based on their merits? Could it be that the fragmentation resulting of the different form factors is the reason we will never see again a 80% market share for a single vendor? I hope so!

A router crashed, the websites have been slowing down

2012-01-27T12:04:47+01:00

If you noticed a slow down in the past minutes, one of the routers of our provider had some issues. This slowed down the services for a short period of time. As you can see on the following graph, suddenly our GET requests to monitor the response time of the services went bong. 20 second response time, this is the equivalent of dead...

But I must say, this is where I am really pleased by our provider, OVH. They immediately explained what was going on: a card of the router crashed hard or died.

Instability of Cheméo

2012-01-18T21:01:46+01:00

At the moment, we move a bit the VMs (Virtual Machines) powering up our PaaS (Platform as a Service). The platform has one virtual machine receiving the code updates and then the Cheméo application is updated from it. The application itself is running on another VM behind the web server. The work we do is to disconnect the web server from the application. So, at the end we get :

1 VM with the web server answering your requests;
2 VMs with the Cheméo application servers — they are doing the work like pulling the data out of the database;
1 VM hosting the code and building the new application code which is then deployed on the previous 2 VMs.

This seams a bit complicated but the advantage is that it is very easy to add new application server VM to handle more requests.

Anyway, the split requires moving component here and there and synchronizing the data, this does not work always as expected and this why, one thing you can expect for sure is a bit of downtime during the next few days.

Migration of Indefero's Backup Server

2012-01-13T09:33:31+01:00

For your information, we are in the process of migrating Indefero's main backup server on our new infrastructure. The new infrastructure has been running for a while and we are satisfied with the stability.

We are going to do at the same time a server upgrade, moving away from Ubuntu and back to the roots, that is Debian. Once the backup server will be up and running smoothly, the main server will follow.

Update: Got a bit of instability at the same time... upgrading here and there an old server is difficult. Time to get the migration to a better system completed!

Cheméo Now on our PaaS

2011-12-31T13:31:03+01:00

As a nice end of the year present, Cheméo is now running on Céondo's private Platform as a Service (PaaS). This is a huge change compared to the previous system. After the normal period of ironing out the system, the infrastructure will be more robust and flexible to allow fast iterative development. The goal is simple, Cheméo must become the reference for chemical engineering data. We are not targeting biological activity but only chemical and process engineers. 2012 is going to be fun.

Launching Cheméo's Labs

2011-12-21T08:21:50+01:00

Few days ago Cheméo's laboratories went life. The labs are running software experiments in the field of chemical and physical properties. They are kind of sandboxes where ideas can be tried without disturbing the main Cheméo website.

The labs are running on top of Céondo's private Platform as a Service (PaaS). This platform will soon host all the services we deliver, from our products Cheméo and Indefero to simpler websites like ceondo.com. In case of, a status website will be kept independently using another technology with a different provider. I will soon write a bit more about this private PaaS.

These are exciting times, the best to close 2011 and start 2012.

Use the DNS to Announce Your ZeroMQ Services

2011-12-11T11:37:45+01:00

If you are building your private PaaS and use ZeroMQ you will end up with the issue of distributing the ZMQ end points of your services to your application. You can use a broker approach, but then, what if the broker is moved to another node? It means that you always end up with a situation where at least a minimal set of service entries need to be distributed to your applications and workers.

The simplest approach is rsync and a text file. Just put everything in a text file and copy it to all the servers. But then you need to manage the list of servers and be sure that each server get the latest version — even if you started the new server at the time you distributed the new version... This starts to be complicated, this is why the simplest approch is simply using the DNS infrastructure with the TXT record.

If you are using djbdns, you can simply add lines like this in your data file:

*.s.myapp.net:192.168.1.100:86400
'myservice.s.myapp.net:192.168.1.123\0729976;192.168.1.124\0729976:3600
'otherservice.s.myapp.net:192.168.1.123\0729976;192.168.1.124\0729976:3600

Now, the day you want to access the myservice from your application, you simply get the DNS TXT record for the corresponding service.

print_r(dns_get_record('myservice.s.myapp.net', DNS_TXT));
Array
(
    [0] => Array
        (
            [host] => myservice.s.myapp.net
            [type] => TXT
            [txt] => 192.168.1.123:9976;192.168.1.124:9976
            [entries] => Array
                (
                    [0] => 192.168.1.123:9976;192.168.1.124:9976
                )

            [class] => IN
            [ttl] => 86400
        )
)

It is simple, robust and allows you to benefit from all the work done on the DNS — for example:

you can access it from any languages (PHP, Python, Java, etc.) and the libraries are usually very robust;
you can announce a different end point depending of the source ip, for example I am using this to announce the testing end points for the services running on the testing subnet;
your DNS server is nice, you can normally extract a lot of stats from it;
with several DNS servers, your system is robust.

Of course this is not to be used as a high speed distributed database, but to distribute service information, this is working perfectly.

Update: Someone nicely pointed out that a SRV record can be used too. Totally right, but in my particular case, I store a JSON string with more than just the end point definition, this is why I need the TXT record flexibility. But if SRV fits your requirements, you should jump for it.

Update: For your information, this is effectively to power Cheméo.

Improving the Response Time of Indefero

2011-12-02T17:35:14+01:00

Improving the speed of Indefero is challenging as it requires managing a lot of moving parts, from the git/subversion backends to the database. This week, I have been working on setting up Graphite for the infrastructure. This is working pretty well and provides graph like the following one.

This graph is extracted from a special Nginx log format which includes the time needed for Nginx to send the response back to the client. The only thing missing is that when I see a spike, I need a way to directly access the corresponding logs to figure out why. At the moment, there are no integrations between these metrics and the logs.

To improve a system, one needs to know the current state. Graphite is a bit hard to setup, but afterwards, it is really easy to push data in. A really nice tool.

The Problem with Performance Logging

2011-11-30T08:32:02+01:00

To run a service like Indefero, you need to log a long list of metrics to follow the load on the system, find the bottlenecks and predict the future needed capacity. To do that, a very powerful system is Graphite, the only issue is that it is only storing and graphing numerical values. Of course, you cannot do different, but the problem is: correlation.

Basically: Once I see that every now and then component is not performing well, how can I drill down in my data to find the reason?

Graphite tells you: this day from 14:05 to 14:07, the rendering of a git tree view was slow. Good to know, the following question is of course: why? If you store more metrics, you can maybe find that I/O was slow on the server X, you can graph together many metrics and visually correlate them. But then, why was I/O slow?

At this point, you need to go one level deeper and take a look at the logs coming from server X from 14:05 to 14:07. This can bring you up to the application level where you figure out that a client repeatedly accessed a page which triggered a git command with a large output, thus loading the server. But to do that you need to access the logs too.

So, Graphite is wonderful, but what I need is that after identifying the subsystem and time range where we have an issue, being able to simply scan through all the corresponding logs in the time range. This would be a kind of integration between Graphite and Graylog2.

My problem now is that Graylog2 is overkill. That is, it tries to provide full text search on the logs, the result is that it requires a very big machinery where I just need aggregation of the logs and the equivalent of a time base search range with a filtering by component, for example webapp.backend.git.

This annoys me, I do not want to build a system by myself.

Small downtime of Indefero

2011-11-16T12:06:14+01:00

Around 11:14 UTC today one switch of the private network went down and required a reboot. The problem has been solved but this resulted in a downtime of about 8 minutes.

Note that this is the issue with the new database server, if the link between the application server and the database server goes down, then the service is down. I will contact the support staff of OVH as in my understanding, they had a kind of redundant system to not rely on a single router.

Database Migration Starting & Done

2011-11-11T09:35:05+01:00

Hello, just to let you know that today November 11, the database migration is starting. You can check this blog post for updates. Here are the steps I will be performing:

test of the web application software upgrade. 09:55 UTC - Done.
shutdown of the PHP processes to prevent update of the database. 09:56 UTC - Done.
force the creation of a PostgreSQL WAL to have consistent backup. 09:57 UTC - Done. Now waiting for the log to be shipped to the warm standby. 10:00 UTC - Done.
start of the warm standby as a new master. 10:04 UTC - Done.
check that the new master is consistent. 10:21 UTC - Done.
update the web application configuration to access the new master. 10:23 UTC - Done.
start the PHP processes again. 10:25 UTC - Done.
immediately clone the new master to have it log ship on another warm standby. 10:37 UTC rsync in progress
10:52 UTC - Done, with a warm standby in another datacenter nicely getting the updates from the master.

This is a bit of cascading but it will always keep several version of the database running and it will always be possible to revert to the original DB server in case of problem.

Last update: The system is now insanely more responsive, pleasure to use is back! If you notice anything unusual, please let me know as soon as possible.

Welcome Decane a New SSD Powered Database Server

2011-11-08T07:28:52+01:00

The Decane is a simple molecule but also the name of our new database server. It is a 24GB RAM/240GB SSD server with a lot of power to provide blazing fast data processing. In the next few days it will go through the standard Ganeti setup and the Indefero PostgreSQL database will be migrated over. Depending of the performance, we may migrate more database VMs on it.

This is the first time I am putting a server with SSD drives in production. I have been an heavy user of SSD drives for my desktop/laptop systems in the past two years and I must say, I will never go back to traditional drives, but of course, the amount of data stored is not the same for desktop and for a server.

So, yes, performance increase of Indefero is on the way!

Update: Decane just joined the Ganeti cluster:

# gnt-node list
Node             DTotal  DFree MTotal MNode MFree Pinst Sinst
node1.ceondo.net   2.7T   2.2T  11.8G  4.0G  8.2G     4     0
node2.ceondo.net   2.7T   2.4T  11.8G  2.9G  9.2G     5     0
node3.ceondo.net 194.3G 194.3G  23.6G  147M 23.4G     0     0

Update 2011.11.09: Base backup of the postgresql database is on the way, this is a huge rsync job and this is of course slowing down the system. Please be patient... thank you.

Update 2011.11.09 20:00 UTC: Ok, the new server is now acting as a warm standby for the database, this will allow fast "failover" to the new database server after the testing period.

Update 2011.11.10 10:28 UTC: The main application server will be unavailable for a short amount of time the time to connect it to its virtual LAN to communicate directly with the warm standby over a private network. Done.

Update 2011.11.10 11:46 UTC: Now that the connection at the switch level is supporting the VLAN, it needs to be configured at the host and vm level. This will again trigger short downtime here and there.

Update 2011.11.10 12:50 UTC: Ok, now setting up a second warm standby which will take over the current one on the new SSD powered server once it will start to act as master. Done.

Update 2011.11.10 13:34 UTC: Ok, things are running as expected, around 21:00 UTC today, Indefero will be stopped for about 15 minutes, this will ensure that we have the warm standby with the latest version. The warm standby will be brought online as master and then the web app will connect to the new master on the new server. Immediately, I will start to populate the new warm standby. Basically a bit of cascading.

Update 2011.11.10 19:20 UTC: Too tired to do the cascading, it is never good to do so when not really fresh. I will perform it tomorrow, ok it will be during the day, but it will be only about 15 minutes of downtime. So expect a downtime of maximum 30 minutes between 09:00 and 12:00 UTC on November 11.

Major Upgrade of the Hosted Indefero Version

2011-10-11T08:42:16+02:00

In the past 24h, I have been testing the latest developer version of Indefero on projects.ceondo.com and I must say, I am pretty pleased by the stability of everything. Congratulations to all the developers involved in getting this done. The only problem is that it requires some heavy migrations and update of the database storage and I must say, I do not like that.

So, some read only tests will be done tonight to check how this can be done without disturbing you too much and if things are going well, the upgrade will be performed. The upgrade can take a very long time. Yes, upgrading 3000 databases is a bit painful.

Anyway, if you take at the new version on projects.ceondo.com, I suppose you will accept this small annoyance as it will bring you a lot of goodies.

Network Maintenance

2011-09-27T07:45:35+02:00

Today, the 27 of September, around 23h00 Paris/Berlin time some of the background processes of Indefero will not run. Our provider is performing some network maintenance.

Basically, they are rewiring our private network, which means that the background communication between the components will be cut. This also means that some website which have data routed over the private network will not function. The total maintenance time is expected to be about 15 minutes. Thank you for your comprehension.

Amazon is too Fast at Delivering Making Kindle Books too Expensive

2011-09-23T12:46:53+02:00

Here in Germany, Amazon is extremely fast at delivering the books I order. In fact, they are too fast to make Kindle version of the books interesting.

As a book customer, here is my experience with Amazon when I buy a physical book, the ones I can store in my personal library:

Tuesday evening, read something about an interesting book, go on amazon.de, order the book.
Wednesday morning, small email telling me the delivery is on the way.
Thursday morning, the bell rings, the postman gives me the book.

Average elapsed time: less than 36h.

Now, the thing is, at home, we have hundreds of books, maybe in fact thousands, but especially, I always have 3 to 4 books on the way, that is, I am usually never without a book to read. So, the incentive to read right now something is pretty low, waiting 36h is not an issue at all.

Now, when looking at the options I have from Amazon when I want to buy a fairly old but popular book:

Basically, the Kindle edition is at €7 and the paperback edition at €10. So why buying a Kindle edition?

Full content of the book.
Start to read right now.
Nicely virtually packed in digital form.
I save €3.

But do I lose doing so?

The content of the book is only licensed to me, I do not own it really.
Not possible to share the book with some friends without a Kindle.
Not possible to write in the book.
Not the same satisfaction at the end of reading the book.

This last one is interesting, I have been reading a couple of books with my Kindle and surprisingly, at the end, the feeling of accomplishment and satisfaction is just not really there. It feels awkward being at the end of the book. With a real book, I can feel the end, close the book and looking at the physical item, reflects about the reading experience. The physical item is embodying the reading experience, it is an anchor, I see it next to my bed and it triggers ideas, makes me automatically reconnect with the story without even opening it. With the Kindle, I just see a Kindle, with so many memories as I combine my Kindle with Instapaper that at the end, it is emotionless. But emotions are my triggers to remember things.

So, at the moment, the delivery speed of Amazon combined with all the loss of the Kindle edition are tilting the balance to buy the paperback edition.

And you, are you reading eBooks? Or do you still prefer the old paper way delivered to you in a snap by the incredible Amazon infrastructure? Also, do you have the same satisfaction, reading a good eBook or the same as paperback?

My request for Amazon: Give me the Kindle for 2€ on top of the paperback edition and cancel it once you receive the paperback delivery notification. Yes, I would pay more but I would have the best of both worlds, directly.

The Git Survey 2011 is Open

2011-09-19T18:48:34+02:00

The Git User's Survey 2011 is now up!

Please devote a few minutes of your time to fill out the simple questionnaire; it'll help the Git community understand your needs, what you like about Git (and what you don't), and overall help us improve it.

The survey will be open from 5 September to 3 October 2011.

The results will be published at GitSurvey2011.

Thanks a lot for your help!

Busy in the Background

2011-09-19T07:54:25+02:00

It may seems that we are relatively quiet at the moment and yes we are, in fact we are like little bees, working hard without making noise.

Maybe you are not aware, but the hosted Indefero offer is running a single Indefero installation for the 3000 forges we are hosting. What is happening is that when you request a page of a forge, the domain is matched against a list and automatically the configuration is set to use the right repositories, the right database, etc.

When the system was setup, it was following the: "Do the simplest thing which can possibly work." Nice, but now we are suffering the pain of growth as the simplest way to go was te build everything in a pretty monolithic system. Ouch.

So, we are hard at work on getting a new system setup to break everything in more manageable components. This is a long haul project, but this is needed to recover our agility in getting new versions online several times a day.

The idea is to do all the changes without changing anything, that is, everything will be done without changing the system for you, then once the migration to components will be completed, new features will be rolled in. Small incremental changes are better on the long run.

This year Fall and Winter are going to be very busy seasons.

Thank You Barclays Wealth

2011-09-02T11:33:11+02:00

It may seems a bit strange as a consulting company to openly say thank you to our bank, but after 4 years of banking with Barclays Wealth, I will never go back to another bank for a UK business. Céondo Ltd, even so it is based in UK, is managed a bit differently than the standard company.

As I am French, moving from one country to another on a semi regular basis, I needed a way to have a company with a stable address. My choices were simple :

France as I am French;
Germany as I was living in Germany at that time;
UK because I already created a UK limited back in 2000.

France was ruled out pretty fast, basically you need to travel every single day of the year to some kind of authorities to get some paper work done. Running a French company from outside France is a recipe for disaster. Germany, with the traditional GmbH was also ruled out, you need to live in Germany to setup one without too many problems, but the main problem is that if you are defacto sole manager of the company and if a patent troll hits you, your company is dead, but the state will hit you again in the civil court to prove that you made an error and all your personal belongings will be used to feed the troll. I suppose the idea behind that was to prevent abuse of the GmbH status, but for me, in the software business in some way or another, this is a no-no.

So, the friendly UK was left. My accountant advised me to contact Barclays Wealth. It took me more than a week of hard work to get all the papers in order with recommendations from my bankers in France, Germany and Denmark, but at the end, Céondo Ltd got an account with them.

Best decision ever. In 4 years, they were always available, nice, polite and efficient. You never hit a call center, you always speak to UK people, they answer the phone within 30 seconds without the need to go through an insane dispatching process. Yes they are expensive, but you know, I would happily pay even more to keep them.

So, I heavily recommend them, it is not easy to go in but it is worth it.

Welcoming Bayer As New Customer

2011-03-15T13:47:54+01:00

This is a great pleasure to welcome Bayer as a new customer. Bayer being big, in this case, I am talking about Bayer Technologies and Services also known as BTS.