• Ссылки на выходные #30

  • Thoughts on UI tests

    Generally, I’m a big fan of writing test. They are great: once you add them, you cannot break logic silently anymore. The time you’ve spent on writing tests pays off completely in that manner you won’t need to fix production on Friday’s night. Tests cannot guarantee your code is free from errors completely, though. But having properly designed tests greatly reduces the number of potential mistakes.

    For a long time, I’ve been developing only server-side tests whereas the UI behavior was tested manually. That has been worked until the moment when we understood we break the UI too often. That’s how the question about having UI tests has appeared.

    TL;DR: I didn’t anticipate maintaining client-side tests (also known as integration test) would be so costly. Yet I managed to get stable UI tests, it took long time of debugging, reading the source code of webdrivers and Google/StackOverflow surfing. Here are some random thoughts on the subject.

    UI tests are complex

    The first issue you might face when dealing with UI tests (and it has never been written clearly in any documentation or manual) is they are unstable and flaking by their nature. In technical terms, they are fragile and difficult to maintain. Testing UI involves lots of technologies and protocols. Under the hood, there is an entire stack of nested calls. Let’s consider a typical PHP/JS-driven web-application with Selenium-based tests. Now follow the testing pipeline.

    Before you run a test, you should launch the application server (probably your REST backend) and Selenium server. Then, once you’ve started a test, it sends request to Selenium to start a new browser session. Java spawns a new process of webdriver – a special program that helps to automate browsers. The driver tries to discover a browser installed on your machine. If everything is alright, the browser process starts. Just a note: you have not done anything important yet, but there are four processes are running already (the app, Selenium, webdriver, browser).

    Now then, every time you call for something like

    >>> driver.find_element_by_id("my_button")

    , it turns into a stack with the steps look like approximately the following:

    1. your PHP/Python/whatever code calls Selenium server.
    2. Selenium remaps the incoming data and calls webdriver server.
    3. The webdriver reads JSON body and sends corresponding binary commands to the browser via socket channel.
    4. The browser performs those actions with some delay. There is no guarantee about timings, the numbers depend on your OS, number of cores, amount of open tabs and so on.
    5. If everything is alright, the data flows in the opposite way: from the browser to the webdriver, to Selenium and finally to the app.

    Remember, it was only a simplest API call. More complicated API expand into series of calls and success only when all of sub-calls managed to finish successfully.

    Versioning hell

    What exactly may occur during the process? Well, everything.

    The most obvious thing is the more software you are trying to work together the more unstable the system becomes. That might not be a problem when all the components are under your control. Say, if it was a micro-service architecture with each node developed by your people. But if fact, they all come from different companies with different points on the subject. Your testing library is written by some enthusiasts. Selenium is the second group. The browser is the third one and they do not listen to anybody, that’s for sure. Developers who have brought the webdriver are the fourth group.

    Now imagine that every product has its own version, release schedule and roadmap. And the version 4.2.1 of this requires at least version 3.2.1 of that. In the release 5.4 the feature X doesn’t work, you should use 5.5-beta instead. And more many of that. These bits of information are never shared in the documentation, but only in comments on Github.

    It reminds me The Planet Parade sometimes when the system performs only if some special version’s numbers are set together. Otherwise, it falls apart.

    They run for too long

    Another issue with integration tests that you might not be aware about is their long execution time. It becomes new to me after long period of maintaining server tests that used to pass in a minute or two. Even having a small set of integration tests, it would take much more time. If we care about running each tests independently without affecting them each other, you need to turn the app to its initial state. That involves resetting the database, logging out, wiping session, preparing local files and so forth. It takes sensible amount time all together. Be prepared your CI pipeline becomes more time-consuming.

    Referencing a missing element

    What I’d like to highlight more in that topic is a strange error named “element stale”. It will definitely rich you when dealing with single page application (aka SPA). It is so common that the official Selenium site has a dedicated page and an exception class named after if. The sad thing, those text is quite stingy on describing what could be the reason of getting such an error. And more important, how to prevent it appears. Hopefully, after long debugging sessions, I’ve got answers to both of those questions.

    Inside browser, every DOM element has its own unique machinery-wise identifier. There is no a single standard of representing it. Depending on browser, you might get either 0.950889431100737-1 or c1ee22f1-b96e-5245-bd85-9e56e1781cbd for the same button. But it does not matter. What really does is it’s quite easy to refer an element that has been removed from the DOM tree when the app page has been re-rendered.

    That’s the root of the problem: such modern frameworks as React, Vue or whatever else continuously update the page components inserting new HTML nodes. Even when you have a component with a button with unique “id” attribute and it has been re-rendered, an new DOM element instance will be created. From the browser’s prospective, it will the another node, completely different to the previous one.

    Each modern testing framework has high-level functions to operate on page elements without knowing their IDs. Consider Python examples:

    >>> driver.click_on_element("my_button")

    In fact, this API call expands into at leas two low-level Webdriver API calls, namely:

    1. Try to discover an element ID by the search term.
    2. Try to click on that element.

    Looks well, but here is what takes place under the hood. You’ve got a long ID of a link or a button, that’s great. But in a millisecond before you clicked on it, the UI has been re-rendered. The reason of that is not so important: maybe, some event has been triggered, or a user hovered some element when moving mouse to it. Or maybe, the framework’s internal algorithm just decided somehow the content should be updated. That means, the original button element you’ve got the ID for was detached from the DOM thee and moved into a temporary array or any other kind of data structure to wait before it is wiped completely from memory. Instead of it, a new DOM node that looks exactly the same took those place. But willing to click on the button, use send the outdated ID. There is no sense in clicking on such kind of elements that are almost dead. The browser returns corresponding error message that expands into Selenium exception.

    I wish that paragraph was on those page. It would save me a week maybe or even more.

    A good solution would be to implement some kind of transactions in your code. Let every complex API be implemented with try/catch clause with some sensible strategy to re-call it in a case of failure, say a number of times or by timeouts.

    High Hopes

    OK, let’s have some final thoughts on the subject, some kind of summary. Testing UI is really far from the server-side ideal world. They are unstable, difficult to develop and maintain. There a lots of undocumented traps you may suffer from during the journey. But still, there are definitely some good things happening nowadays.

    First, the official Webdriver protocol is developing actively. This protocol is subject to substitute the outdated Selenium JSONWire protocol. It has become a part of W3C so I hope it will be developed so forth.

    More Selenium-free libraries appear on Github: Nightwatch, Capybara, Puppeteer… That is great because having the only one monopolist in such specific area stops the progress.

    But there are lots of things to be done. What I’d like to see at first place is some kind of cookbook with common advise and good practices on how to test UI without suffering. Those bits should not depend on specific language or library although there could be platform-specific sections. I have never met such kind of a cookbook across the Internet; please correct me if there are any of them.

    When writing a library for browser automation, it would be great to not only cover the low-level calls but also design an abstract layer of high-level API to make it easy to use by those who are not familiar with all the issues I mentioned above. Say, a high-level function (or method if we talk on OOP terms) that clicks on a button should take into account the case when the element is stale. From my prospectives, such a functional language powered with immutable data structures and homoiconicity as Clojure (or any Lisp-family one) would be a great choice.

    Ok, let’s finish for now. Thank you for listening out. Next time, I’ll say more on technical side of testing code.

  • Mozilla makes me crazy

    What I’d like to share with you is my suffering from dealing with browsers. I’m an author of a Clojure library that automates browsers. Briefly, the library doesn’t depend on Selenium since it implements the Webdriver protocol with pure Clojure. It brings flexibility and freedom in the way you’d like to build your API.

    At the same time, everybody who is willing to tame several browsers at once is doomed to open the Pandora box. There is the official standard I thought, so definitely everybody will follow it. A naive laddie I was.

    I don’t know if there is some kind of curse or black magic, but sometimes developers cannot just implement the standard as it dictates the things should be done. For example, it says the server should accept a value field. Developers write code that takes text instead. When I see it, I have strange feeling of being completely stunned without any understanding of what’s going on here.

    But what exactly I wanted to discuss is how does Mozilla develop their software.

    Briefly, they break everything apart moving further. And I don’t know how to deal with it. Maybe they enjoy coding in Rust so much that their mission to ship good software has faded away.

    I know I’m not a Mozilla engineer and would never be capable of joining them, but still. As a user of their products, namely Geckodriver and Webdriver utilities, I consider myself being right when criticizing them.

    With each next release of Geckodriver, it behaves worth and worth. When I first started developing the library (a bit less then 1 year ago), it was 0.13 version available and it worked fine. I wrote a bunch of unit tests that all passed. Nowadays, the latest version is 0.19 and it just doesn’t work.

    Yes, you heard correct, it does not even respond to any API call:

    ~/Downloads> ./geckodriver-0.19.0
    1510326430666 geckodriver INFO geckodriver 0.19.0
    1510326430680 geckodriver INFO Listening on
    curl -X POST -d '{"desiredCapabilities": {}}' ""

    Calling curl utility throws a long stack trace saying something about sandbox restrictions without any word on how to fix that or at least where to look for help.

    Here is a related issue that confirms I’m not the first one who faced it. It’s closed! “As I said, this isn’t a problem with geckodriver”, one of developers says. OK, but I’m still curious about why does the version 0.13 work fine whereas switching on 0.19 leads to failure? Proof:

    ~/Downloads> ./geckodriver-0.13.0 --version
    geckodriver 0.13.0
    ~/Downloads> ./geckodriver-0.13.0
    1510327215587 geckodriver INFO Listening on
    curl -X POST -d '{"desiredCapabilities": {}}' ""
    >>> {"sessionId":"e744bbdd-1b3f-9249-827f-02204bbc81c8","value":{"acceptInsecureCerts":...

    Ok, let’s decrease the version a bit. I downloaded them moving backwards in time. Again, 0.18 still does not work. We need to go deeper. The previous one numbered with 0.17 starts well, but suddenly, I cannot fetch a new session with my library, it just returns nil value. That’s strange.

    Going down to 0.15. Don’t know why, but I cannot fill any input field, the API fails with a strange error saying I didn’t pass the “text” field. Hm, I clearly remember that the API accepts “value” field. That’s what the W3C standard says. Geckodriver 0.13 follows it, but not the 0.15 release! After fetching the official Mozilla repo and searching in its history for a while, I found this (truncated):

    diff -r 225d1faf513b -r b429bf0078c4 testing/webdriver/src/command.rs
    --- a/testing/webdriver/src/command.rs	Tue Mar 07 18:50:56 2017 +0000
    +++ b/testing/webdriver/src/command.rs	Wed Mar 15 13:54:19 2017 +0000
    @@ -752,7 +752,7 @@
     pub struct SendKeysParameters {
    -    pub value: Vec<char>
    +    pub text: String
     impl Parameters for SendKeysParameters {
    @@ -760,26 +760,14 @@
             let data = try_opt!(body.as_object(),
    -            Ok(chars[0])
    -        }).collect::<Result<Vec<_>, _>>());
    +        let text = try_opt!(try_opt!(data.get("text"),
    +            text: text.into()
    @@ -787,10 +775,7 @@
     impl ToJson for SendKeysParameters {
         fn to_json(&self) -> Json {
             let mut data = BTreeMap::new();
    -        let value_string: Vec<String> = self.value.iter().map(|x| {
    -            x.to_string()
    -        }).collect();
    -        data.insert("value".to_string(), value_string.to_json());
    +        data.insert("value".to_string(), self.text.to_json());

    Hold on, what was the purpose to change that field? Everything was working, right? Who asked for that? Did the standard change? No, it’s still the same! Why have you guys just changed the API? You could easily rewrite the code without renaming “value” field into “text”.

    Listen, I’ve already got three different versions of those API for Chrome, Firefox and Safari. In addition to that, your force me to switch on version inside Firefox branch turning my code into if/else hell.

    From your perspectives, what’s the difference in processing either “value” or “text” field? But for me, you’ve broken my software! It doesn’t work anymore.

    I could not find a diff that would proof the fact of moving “sessionId” field because the repo’s history is pretty complicated. But it’s easy to check that. The version 0.17 returns:


    whereas 0.13 returns:


    Look, they just wrapped the whole data into “value” subfield shifting it one level below. Again, what was the purpose of doing this? Did you think of those users who have already been using your code for some time?

    Thanks to those weird changes, I need to refactor my code just because Mozilla guys decided to rename something. And more over that, thank you for not sending back the driver’s version in API responses. It would be not so challenging when having it.

    I wonder why has Chromedriver been working all that time without errors? It’s written in C++ and Python and the code is quite simple and linear. And it’s much stable then its Rust-driven rival.

    OK, it’s time to summarize all together. I don’t think that anybody will share my misery on the subject. The Webdriver spec is quite specific thing, so who would really care…

    But the main point of that talk is you should never break backward capability even at the gunpoint. Just keep things working, that’s the main priority when you change something. Extend, but not truncate your product. And really, just use it sometimes in a way that an ordinary user does.

  • Последнее видео с Рефакторинга

    Клуб Рефакторинга живет и здравствует, хоть я и забываю репостить видео в блоге. Вот пачка последних выступлений:

    Ссылки на слайды в описании к каждому видео. Больше записей на нашем Ютуб-канале.

  • Silent software

    I really value such kind of software that does not interrupt you. It should have happened to you I believe: you open a notebook when thouthands of notification balloons appear above the lock screen. Icons are jumping, everything is blinking. Or you open a program that immediately shows daily tips, asks for updates and so forth. Once you’ve closed the last tooltip, you start recalling what were you about to do with that program.

    That was the reason I abandoned Firefox. Every its extension, when updated, opened a new tab with release notes. So every time I opened a browser I had to close those tabs I didn’t ask to open.

    I’ve been fighting with that as I can for a long time. I turn off all the notifications on both laptop and mobile phone. I cut unwilling elements with Adblock and dump those patters into private Git repo. If a website doesn’t have RSS feed I don’t use it. I read long texts only with Kindle.

    I cut YouTube interface completely to see only a search bar and the video player by itself. No recommended sidebar or comment feed.

    When I open my laptop willing to send a email but some program prevents me from that (no matter was is a notification or a sound) it’s a serious reason to stop using it. If any site sticks a city selection or email subscription dialog across the whole screen I close it. No worries, I’ll find another one even better.

    A rule of being not interrupted by software as important as a rule of being not interrupted by other people. Everybody has a right to be alone.

    I hope one day that simple thing will occur to those who develop software.

  • Ссылки на выходные #29

    Севодня выпуск про женщин. Гендерное равенство, все дела.

    Любите женщин!

  • Как шарить картинки через Гитхаб

    Илья жалуется на облачные хранилища за то, что не показывают нормально картинки. Нормально значит средствами браузера по прямой ссылке.

    Действительно, это проблема. Картинку, расшаренную через какой-нибудь условный Дропбокс, невозможно открыть без мата. Сначала прогрузится верхняя панель, потом тулбар, скрипты и виджеты, а в конце, может быть, дойдет до картинки.

    Еще хуже обстоят дела с документами. Интерфейс рисует их прямо на странице: медленно, с тормозами, огромным потреблением памяти. Нет глупее поступка, чем шарить файлы через облачные сервисы.

    Я пишу этот пост чтобы, во-первых объяснить причины такого поведения, а во-вторых, предложить достойное решение.

    Ни один облачный сервис, будь то Дропбокс, Гугл-диск, Яндекс-диск, Мейл-облако, не даст прямую ссылку на файл. Виноваты в этом не программисты и даже не менеджеры. Просто сервис с прямыми ссылками становится убыточным и рискует закрыться.

    Если дать пользователям анонимно ссылаться на файл в сети, это мгновенно превратит сервис в хостинг картинок для форумов и борд. А это противоречит бизнес-целям сервиса. Ведь он задумывался не как хостинг, а хранилище личных данных. Возникает разница между целью и результатом.

    Может, вы помните, в Дропбоксе была особая папка public, содержимое которой видно любому анониму. Пользователи шарили из нее картинки для блогов и соцсетей. Особо одаренные держали там статичные сайты. Какой-нибудь условный Вася постил с Дропбокса смешную фотку на Пикабу, и сервис раздавал ее сотни тысяч раз. Теперь представьте, что фотка не одна, а сто, и таких Вась тысячи.

    Дропбокс, как известно, хостится на Амазоне и выплачивает огромные деньги за трафик. При этом им нужно окупить расходы на инфраструктуру, офис и Гвидо Ван Россума лично. Поэтому минимизация трафика и запрет на прямые ссылки – неизбежное следствие.

    Конечно, открывая картинку в интерфейсе Дропбокса, я всецело на стороне пользователя. Но другой частью разума понимаю, что шаринг по прямой ссылке превратит сервис в файло-помойку.

    Внезапно, самый удобный способ пошарить картинку сегодня – это Гитхаб! Для этого не нужно пихать файл в репозиторий. В Гитхабе, кстати, не дураки сидят: все статичные медиа-файлы из репозитория они раздают с заголовком Content-Disposition: attachment, что говорит браузеру скачать файл, а не открыть для просмотра.

    Однако, если залить картинку в любой раздел проекта, например, ишью, пулл-реквест, вики или комментарий, она будет доступна по прямой ссылке без каких-либо проблем, анонимно и круглосуточно. Достаточно перетянуть файл в текстовое поле, картинка тут же подгрузится, в тексте появится ее уменьшенная копия со ссылкой на оригинал.

    Я иногда пользуюсь этим способом, если в тексте много фотографий. Блог у меня на Гитхабе, однако заливать в репозиторий много бинарных файлов нехорошо. Поэтому я создаю в проекте блога ишью с именем, например “Baltimore photos”, забрасываю туда фотки и копирую ссылки. Посмотреть в действии можно в моем рассказе про Балтимор. Обратите внимание на адреса картинок. А вот ссылка на тот самый ишью с фотками.

    Я не скажу точно, какие ограничения накладывает Гитхаб на такой хостинг. Скорее всего, система учитывает домены и считает трафик, но еще ни разу не сталкивался с ограничениями. Конечно, этот способ не подойдет для аудитории какого-нибудь Варламова. Но для блога и небольших сообществ – вполне.

    Наконец, Гитхаб – отличный способ организовать рабочий процесс. Не кидайтесь картинками через Телеграм. Заведите проект, в нем тасочку, добавьте метки, ответственных. Все файлы и комментарии должны оседать в таске, чтобы через год открыть и все вспомнить.

    Предположим, студенты должны прислать работы к конкурсу. Создайте проект в Гитхабе (приватный, если нужно). Каждый участник открывает таску со своим именем, подливает файл. Он доступен по прямой ссылке. Все комментируется, трекается, хранится неограниченно долго, на почту приходят письма.

    Мне кажется, проще и удобней сегодня ничего нет.

  • Егор Бугаенко в Глубоком Рефакторинге. Анонс

    В декабре в Воронеж приедет Егор Бугаенко – известный разработчик, евангелист, автор книг. Егор выступит в клубе Рефакторинга с двумя докладами на темы разработки, собеседования и карьерного роста. Встреча будет полностью посвящена визиту Егора.

    В перерывах и после мероприятия зрители смогут пообщаться с гостем в неформальной обстановке и задать вопросы. Будет трансляция, позже – видео на нашем канале и Егора с английскими субтитрами.

    Короче, это уникальная возможность повидаться с известным разработчиком. Если вы в Воронеже и как-то связаны с ай-ти, приходите обязательно и поделитесь новостью с друзьями.

    Тем, кто слышит о Егоре впервые, следует почитать его блог и посмотреть выступления на Ютубе.

    Поскольку гостя такого ранга мы принимаем в первый раз, коллеги запустили гугло-форму с целью прощупать почву. Прежде всего нас интересует число участников, чтобы не пришлось под Новый год носиться в поисках помещения.

    Напомню, чат Рефакторинга у нас в Телеграме, есть вопросы – ответим.

  • Elegant Objects


    I’ve just got Yegor Bugayenko’s “Elegant Objects” book. Even without reading a single page of it, I can definitely say the book worths doing it. How do I know in advance you may ask?

    That’s simple: it’s got great design. You may prove it too when opening the book at somewhere in the middle of it and scan the page.

    How does an ordinary IT book look like? Usually, its design is full of details made without any sense. A typical O’Reilly book makes me feel like a kid: it has lots of different paragraph styles, borders, lines, gray bars. Before you start reading, there is a legend with up to five icons. Look, that icon means to be careful, this one stands for “experienced users only” and so forth. Every page carries the author’s name, the title of the book and the chapter caption as if I really need to keep all of that in my mind constantly.

    Instead, the Yegor’s book are made of high quality design. Every page has only text but nothing else. No lines, bars or icons. There only two text stiles per the entire book, one is for ordinary text and the second one is for code. Again, the code is not put into the colored bar or whatever. It’s just text, and this is amazing.

    Except the main text, there is no any information on a page but its number. I really appreciate such design because it servers the only thing it was aimed for – to encourage me to read the book (but not to buy and put it on my shelf).

  • Без ORM. Доклад в Глубоком Рефакторинге

    Выступил на последнем митапе с докладом про ORM. Редкий случай, когда самому понравилось. По этой причине решил поделиться с вами.


    Планирую написать расшифровку на английском.

Страница 31 из 66