Ivan Grishaev

Ivan Grishaev's blog

Writing on programming, education, books and negotiations.

Home About Bookshelf RSS
Clojure Emacs Python Programming Interview Video
Книга «Clojure на производстве»
Книга «Clojure на производстве», второй том

  • Conditional Queries in Datomic

    , ,

    Let’s discuss one thing related to Clojure and Datomic that might be a bit tricky especially if you have never faced it so far.

    Imagine you return a list of some entities to the client. It might be a list of orders, visits or whatever. Usually, when a user is exploring a long list of something, it’s a good idea to let them filter the list by some criteria: price, date, age etc. On the server side, you need to apply those filters conditionally. Say, when the date_from query string parameter is passed, you apply it to the query as well or live it untouched otherwise. The same for the rest of filters.

    Things become more tough when filters are applied to foreign entities. For example, an order references a user, and a user references a department. If the department_name parameter has been passed, you should join all the required tables and filter a proper field.

    Joining all the tables even if no filters were supplied is a wrong approach. Join operations are expensive and should never be performed in vain. A toolset that joins tables should take into account which ones has already been added into a query and never link them twice.

    Such systems that control the cases mentioned above are names ORMs. They are pretty complicated and full of ugly code and implicit hacks. But on the top level they behave quite friendly. Say, in Django (a major Python framework) I would perform something like that:

    query = models.Order.objects.all()
department_name = request.query_string.get("department_name")
if department_name:
    query = query.filter(user__department_name=department_name)

    Looks pretty neat. This automatically joins users and department tables under the hood using inner join SQL clause and puts department.name = ? condition into where section.

    People who work with such non-wide spreaded languages as Clojure usually do not use ORMs. But still, we need to build complicated queries. The community offers a handful of libraries (YeSQL, HoneySQL, Korma) where a query is being constructed within data structures: vectors, maps. I’ve been always against that approach. Before getting more experienced with Clojure, I felt uncomfortable constructing nested vectors like this one:

    [:select [:foo :bar]
 :from :test
 :where [(= :name "test")
         (when age-param
           (> :age age-param))]]

    The reason why I do not appreciate that is I cannot see the final query behind brackets and colons. It will definitely fail me once I need to express something like this (a fragment of a query from production):

    select
  foo.id,
  array_agg(foo.type_id) filter (where foo is not null) as type_ids,
  foo.name
from foo_table as foo

    Modern libraries say it’s easy and fun to express your queries through data structures, but it is not, really. It becomes a challenge when applying multiple conditions to a data structure without seeing the final result.

    A good approach might be using a templating system. Say, HugSQL library allows to inject Clojure snippets into your SQL query. Those snippets are surrounded with standard SQL comments so they do not break syntax. There won’t be an error if you copy and paste such a Clojure-instrumented query into some RDBS administration tool.

    Here is an example of declaring such a query:

    -- :name get-oreders :?
select o.*
from oreders o
/*~ (when (:department-name params) ~*/
join user u on o.user_id = u.id
join departments d on user.department_id = d.id
where
    d.name = :department-name
/*~ ) ~*/
order by o.created_at;

    Than it compiles into a Clojure function:

    ;; no joins, no filters
(get-oreders db)

;; causes joins and filtering
(let [dep-name (-> request :params :department-name)]
  (get-oreders db {:department-name dep-name}))

    You may go further and include Selmer, a template library inspired by Django. It’s aimed at HTML rendering first but still may be used for any kind of documents including SQL.

    As I see it, a good templating system would be enough to generate SQL that fits your business logic.

    Now I’d like to discuss the same problem when using Datomic instead of classical RDBS solutions. All the tutorials that I have read do not cover a case then you need to apply several filters to a query. Suddenly, it may really become a problem. Let’s return to our example with orders. Once you don’t have any filters, the query looks simple:

    '[:find (pull ?o [*])
  :in $
  :where [?o :order/number]]

    But if you’ve got a department name you need to:

    1. inject a new parameter into :in section;
    2. inject additional clauses into :where section;
    3. prevent joining a user or a department entities twice if you need to filter by other department or user field.

    As we’ve seen, once you have a template system, you may render SQL queries as well. All you need is to write a query, test how does it behaves and then wrap some parts of it with special conditional tags.

    In Datomic, a query is usually a vector of symbols. Moreover, an immutable one. Thus, you cannot modify a query and adding something in the middle of it would be difficult. Surely you could wrap a query into an atom or track indexes where to inject a new item somehow but all of that would be a mess.

    What I propose is using a special kind of a query represented as a map with :find, :where and other keys. As the Datomic documentation says, when processing a query, every vector is turned into a map anyway. If we had a map, it would be easier to inject new items into it.

    To avoid wrapping a map with an atom or redefining it continuously inside let clause, there is a great form named cond->. It is a mix of both threading macro and cond clause. It takes an initial value and a bunch of predicate/update pairs. If a predicate form evaluates in true, an update form is fired using the standard threading macro. Thus, an update form should be either a function or a list where the second argument is missing and will be substituted with a value from a previous pair.

    What’s the most interesting about cond-> is unlike cond or case forms, its branches are evaluated continuously. Each update form takes a value that a previous form has produced. In other terms, an initial value goes through multiple updates without being saved in some temporary variable.

    In example below, I’ve got a data set that consists from user, location and visit entities. Both user and location are simple ones and store just dates, strings and so on. A visit is a bit more complex. It means that a user has visited a location and assigned a mark to it. Therefore, a visit references a user and a location entities as well.

    The goal is to get an average mark for a specific location. In addition, such a value might be filtered by user’s age, gender or location’s country name. Those parameters come from a query string and could be either totally skipped, passed partially or completely. I’ve got a function that accepts a location id and a map of optional parameters:

    (defn location-avg
  [location-id {:keys [fromDate
                       toDate
                       fromAge
                       toAge
                       gender]}]

    The initial Datomic query:

    (def query-initial
  '{:find [(avg ?mark) .]
    :with [?v]
    :in [$ ?location]
    :args []
    :where [[?v :location ?location]
            [?v :mark ?mark]]})

    Now, here is a long pipeline with comments:

    (cond-> query-initial

  ;; First, add two initial arguments: database instance and location reference.
  ;; This form will always be evaluated.
  true
  (update :args conj
          (get-db)                    ;; returns the DB instance
          [:location/id location-id]) ;; location reference

  ;; If either from- or to- date were passed, join the `visit` entity
  ;; and bind its `visited_at` attribute to the `?visited-at` variable.
  (or fromDate toDate)
  (update :where conj
          '[?v :visited_at ?visited-at])

  ;; If the `fromDate` filter was passed, do the following:
  ;; 1. add a parameter placeholder into the query;
  ;; 2. add an actual value to the arguments;
  ;; 3. add a proper condition against `?visited-at` variable
  ;; (remember, it was bound above).
  fromDate
  (->
   (update :in conj '?fromDate)
   (update :args conj fromDate)
   (update :where conj
           '[(> ?visited-at ?fromDate)]))

  ;; Do the same steps for the `toDate` filter,
  ;; but the condition slightly differs (< instead of >).
  toDate
  (->
   (update :in conj '?toDate)
   (update :args conj toDate)
   (update :where conj
           '[(< ?visited-at ?toDate)]))

  ;; To filter by user's fields, we bind a user reference
  ;; to the `?user` variable:
  (or fromAge toAge gender)
  (update :where conj
          '[?v :user ?user])

  ;; If from/to age filters we passed, bind user's age
  ;; to the `?birth-date` variable.
  (or fromAge toAge)
  (update :where conj
          '[?user :birth_date ?birth-date])

  ;; Then add placeholders, arguments and where clauses
  ;; for specific filters: fromAge, if passed...
  fromAge
  (->
   (update :in conj '?fromAge)
   (update :args conj (age-to-ts fromAge))
   (update :where conj
           '[(< ?birth-date ?fromAge)]))

  ;; ...and the same for toAge.
  toAge
  (->
   (update :in conj '?toAge)
   (update :args conj (age-to-ts toAge))
   (update :where conj
           '[(> ?birth-date ?toAge)]))

  ;; To filter by gender, bind user's gender to a variable
  ;; and add a clause:
  gender
  (->
   (update :in conj '?gender)
   (update :args conj gender)
   (update :where conj
           '[?user :gender ?gender]))

  ;; The final step is to remap a query (see below).
  true
  remap-query)

    Remapping a query is important because the initial data is a bit wrong. The proper structure for a map query looks as follows:

    {:query <query-map>
 :args [db location_id fromDate ...]}

    In my case, I believe it’s simpler to keep one-level map rather than deal with two levels (:query first, then :args). It would force me to use update-in instead if just update and write more code. Here is the remap-query function:

    (defn remap-query
  [{args :args :as m}]
  {:query (dissoc m :args)
   :args args})

    Finally, let’s check our results. If somebody passes all the filters, the query will look like:

    {:query
 {:find [(avg ?mark) .],
  :with [?v],
  :in [$ ?location ?fromDate ?toDate ?fromAge ?toAge ?gender],
  :where
  [[?v :location ?location]
   [?v :mark ?mark]
   [?v :visited_at ?visited-at]
   [(> ?visited-at ?fromDate)]
   [(< ?visited-at ?toDate)]
   [?v :user ?user]
   [?user :birth_date ?birth-date]
   [(< ?birth-date ?fromAge)]
   [(> ?birth-date ?toAge)]
   [?user :gender ?gender]]},
 :args [<db-object> [:location/id 42] 1504210734 1504280734 20 30 "m"]}

    Now you pass it into datomic.api/query function that accepts a map-like query. In my case, the result is something like: 4.18525.

    As you have seen, composing complicated queries with Datomic might a bit tricky due do immutability and differences between string templates and data structures. But still, Clojure provides rich set of tools for processing collections. In my case, the standard cond-> has made all the pipeline. Neither atoms nor other tricks to track the state were required. There is a common rule: once you’ve got stuck with a data structure, keep yourself from inventing “smart” ways to deal with it. There is probably a built-in macro in clojure.core for that.

  • Фейсбук прекрасен

    ,

    Каждый месяц повторяется печальное событие: мне нужно зайти в ФБ и сделать анонс митапа. Я тяну время, но часа X не избежать.

    facebook

    Даже не знаю, что может быть хуже Фейсбука: столь тормозной и неорганизованный сайт еще поискать. При том что денег у них до жопы.

    Погрязли в жадности и бюрократизме.

  • In-Memory SQLite Database In Clojure

    , ,

    Recently, I’ve been working with a SQLite database using Clojure. In this post, I’d like to share my experience that I’ve got from that challenge.

    SQLite is a great tool used almost everywhere. Browsers and mobile devices use it a lot. A SQLite database is represented by a single file that makes it quite easy to share, backup and distribute. It supports most of the production-level databases’ features like triggers, recursive queries and so on.

    In addition, SQLite has a killer feature to be run completely in memory. So, instead of keeping an atom of nested maps, why not to store some temporary data into well organized tables?

    JDBC has SQLite support as well, you only need to install a driver, the documentation says. But then, I’ve got a problem dealing with in-memory database:

    (def spec
  {:classname   "org.sqlite.JDBC"
   :subprotocol "sqlite"
   :subname     ":memory:"})

(jdbc/execute! spec "create table users (id integer)")
(jdbc/query spec "select * from users")

> SQLiteException [SQLITE_ERROR] SQL error or missing database
> (no such table: users)  org.sqlite.core.DB.newSQLException (DB.java:909)

    What the… I’ve just created a table, why cannot you find it? An interesting note, if I set a proper file name for the :subname field, everything works fine. But I needed a in-memory database, not a file.

    After some hours of googling and reading the code I’ve found the solution.

    The thing is, JDBC does not track DB connections by default. Every time you call for (jdbc/*) function, you create a new connection, perform an operation and close it. For such persistent data storages like Postgres or MySQL that’ fine although not effective (in our project, we use HikariCP to have a pool of open connections).

    But for in-memory SQLite database, closing a connection to it leads to wiping the data completely out form the RAM. So you need to track the connection in more precision way. You will create a connection by yourself and close it when the work is done.

    First, let’s setup your project:

    :dependencies [...
               [org.xerial/sqlite-jdbc "3.20.0"]
               [org.clojure/java.jdbc "0.7.0"]
               [com.layerware/hugsql "0.4.7"]
               [mount "0.1.11"]
               ...]

    and the database module:

    (ns project.db
  (:require [mount.core :as mount]
            [hugsql.core :as hugsql]
            [clojure.java.jdbc :as jdbc]))

    Declare a database URI as follows:

    (def db-uri "jdbc:sqlite::memory:"

    Our database shares two states: when it’s been set up and ready to work and when it has not. To keep the state, let’s use mount library:

    (declare db)

(defn on-start []
  (let [spec {:connection-uri db-uri}
        conn (jdbc/get-connection spec)]
    (assoc spec :connection conn)))

(defn on-stop []
  (-> db :connection .close)
  nil)

(mount/defstate
  ^{:on-reload :noop}
  db
  :start (on-start)
  :stop (on-stop))

    Once you call (mount/start #'db), it becomes a map with the following fields:

    {:connection-uri "jdbc:sqlite::memory:"
 :connection <SomeJavaConnectionObject at 0x0...>}

    When any JDBC function or a method accepts that map, it checks for the :connection field. If it’s filled, JDBC uses that connection as well. If it’s not, a new connection is issued. In my case, every execute/query call created a new in-memory database and stopped it right after the call ends. That’s why the second query could not to find users table: because it was performed within another database.

    Now with the db started, you are welcome to perform all the standard jdbc operations:

    (jdbc/execute! db "create table users (id integer, name text)")
(jdbc/insert! db :users {:id 1 :name "Ivan"})
(jdbc/get-by-id db :users 1) ;; {:id 1 :name "Ivan"}
(jdbc/find-by-keys db :users {:name "Ivan"}) ;; ({:id 1 :name "Ivan"})

    Finally, you stop the db calling (mount/stop #'db). The connection stops, the data disappears completely.

    For more complicated queries with joins, HugSQL library would be a good choice. Create a file queries.sql in your resources folder. Say, you want to write a complex query that filters a result by some values that probably are not set. Here is an example of what you should put into queries.sql file:

    -- :name get-user-visits :?
select
    v.mark,
    v.visited_at,
    l.place
from visits v
join locations l on v.location = l.id
where
    v.user = :user_id
    /*~ (when (:fromDate params) */
    and v.visited_at > :fromDate
    /*~ ) ~*/
    /*~ (when (:toDate params) */
    and v.visited_at < :toDate
    /*~ ) ~*/
    /*~ (when (:toDistance params) */
    and l.distance < :toDistance
    /*~ ) ~*/
    /*~ (when (:country params) */
    and l.country = :country
    /*~ ) ~*/
order by
    v.visited_at;

    In your database module, put the following on the top level:

    (hugsql/def-db-fns "queries.sql")

    Now, every SQL template has become a plain Clojure function that takes a database and a map of additional parameters. To get all the visits in our application, we do:

    (get-user-visits db {:user_id 1 :fromDate 123456789 :country "SomePlace"})
> ...gives a seq of maps...

    Hope this article will help those who’ve got stuck with SQLite.

  • Пять лет блогу

    То, что я давненько ничего не писал, связано с тем, что залип в конкурсе Highload Cup. Две недели ни о чем другом думать не мог, все пытался улучшить решение. И тут вспомнил, что у блога юбилей: первая запись была сделана пять лет назад, 8 августа 2012 года.

    Речь идет о наивной заметке под названием “Рассылка смс в Питоне”. Там я популярно рассказываю, как слать смс через апишку уже несуществующего сервиса. В те дни я работал в читинском Энергосбыте, писал на 1С, Дельфях и Питоне, а лиспы и ФП были влажными мечтами.

    Пять лет по меркам интернета все-таки срок. Много ребят, которых я читал, со временем забили на блоги, не продлили домены. А мне нравится: блог стал настоящим хобби, которое, надеюсь, продлится всю жизнь.

    За прошедшее время я написал 320 постов. Не так много, но я не гонюсь за количеством. Некоторые заметки, первоначально задуманные как пара абацев, разрастались до нескольких экранов.

    В один момент я поймал себя на мысли, что злоупотребляю низкими приемами: жалуюсь на сервисы, высмеиваю тех, кто не согласен со мной. Это работает, но не в том направлении, в котором бы мне хотелось. Теперь стараюсь писать только на важные темы: программирование, образование, карьера, книги.

    Было неприятно видеть, как любимые мной блоги вырождались в комментарии политических новостей. Эту тему я тоже решил пресечь.

    Решил двигаться в сторону англоязычной аудитории. В России на Кложе пишут три с половиной анонимуса, поэтому нет смысла писать о ней на русском.

    Технические заметки на английском порой дают сильный отклик. После публикации статьи о миграции с Постргреса на Датомик в первый же день ко мне зашли 9000 янглоязычных посетителей.

    После нескольких удачных статей про Кложу меня добавили в агрегатор Планета Кложи. Если в посте будет слово “Кложа” на английском, статья попадет в агрегатор. Поэтому если я не хочу, чтобы такое случилось, пишу на русском.

    В планах сделать на сайте теги, рубрикацию и прочие ништяки. Благо, блог статичный на Руби-движке, исходники на Гитхабе, и я знаю, у кого можно накопипастить. А пока вот просто список постов, заслуживающих внимания:

  • Conceptual languages

    , , ,

    This is a short note on what I’m thinking about programming languages.

    Briefly, I reckon some languages as being conceptual. In a conceptual language, every part of it obeys some global idea. This idea forms design of a language, the way it behaves and develops. If affects libraries, community and ecosystem.

    Take Lisp, for example. The idea is, when we store our code as data, it brings incredible possibilities to process code as data or turn the data into code. No other language may offer something like that, only Lisp does.

    In Erlang, every task is solved within a cascade of lightweight processes named actors. Two actors may communicate directly being run on different servers. Actors do not create Unix threads. They are managed by OTP rather than operation system.

    With Haskell, you’ve got quite strong and flexible type system. Types are the most important part of a typical Haskell program. Once it compiles, it will work for sure.

    Clojure, a modern Lisp dialect, provides immutable data structures and powerful abstractions for concurrency.

    Respectively, non-conceptual languages do not have such a global approach. They try to implement as many features as possible to satisfy every domain: OOP, anonymous functions (lambdas), lazy evaluation, etc. As a result, we’ve got everything but nothing: each part of such a language is not as powerful as its analogies form those ones I mentioned above.

    Take Python, for example. Although it has such basic functional blocks as map, reduce and lambdas, programming with it in a functional way would be a mess.

    Every part of classical Javascript is just ugly.

    Java, a language with static type system, allows you to pass Null instead of any object and end up with NPE exception.

    Although conceptual languages are not perfect, they seem to be easier for me to learn because they have some common rules that could not be broken. Say, in Haskell, you just do not have plain Null value. In Clojure, you cannot modify a dictionary, and so on.

    They cannot be substituted with other languages. Really, how can you substitute Lisp or Erlang? There aren’t any alternatives for them.

    I believe, the future is about conceptual languages. To develop AI or distributed systems, we need something more sensible than yet another language with classes and syntactic sugar.

    I’m not sure it could be Lisp, but something that borrows most of its features.

  • Weekly links #28

    ,

  • Weekly links #27

    ,

    I’ve just returned from EuroClojure, it was amazing! Thank you everybody who brought it to us.

    Here are some interesting links I found on Medium:

  • Берлин. Городские детали

    ,

    Побывал на Евро-Кложе в Берлине и хочу рассказать про город. Я не ахти какой путешественник: езжу мало, не особо наблюдателен. Читая Лебедева или Варламова, я долгое время не понимал, зачем они без конца пишут про урны и тротуары. Подумаешь, плитка лучше – и у нас можно пройти!

    А в этот раз меня проперло на городскую среду. Гулял по городу и офигевал с того, как классно устроен город. Еще больше я недоумевал, как раньше не понимал разницы между плохой городской средой и хорошей. Видимо, после тридцати все-таки проснулась черточка.

    Прежде всего, великолепно уложены тротуары. Это может быть плитка, массивные плиты, булыжник или мелкие камешки, но идти по ним и любоваться паттерном можно бесконечно. Все детали пригнаны как в лего, все по паттерну. Когда плитка не квадратная (или если ее кладут под углом), крайние плитки не режут, а изготавливают специальной формы, чтобы стыковаться с соседними поверхностями.

    Далее, нигде нет открытой земли. О вреде открытых клумб те же Варламов с Лебедевым писали сто раз. Любой клочок земли, необходимый растениям, во-первых, утоплен ниже уровня тротуара, а во-вторых, накрыт сеткой или замурован галькой. У нас же с тротуаров и клумб круглый год сыпется грязь и песок, а дворники соскребают обратно на клумбу.

    Шокировало, что нет борюров. Тротуар полностью бесшовный. Меняется размер и паттерн плитки, булыжник чередуется с плитами, но уровень нигде не перепадает. На дорожных переходах, в выездах с прилегающих территорий, рядом с подъездами, словом – нет бордюров. Я прошел в одном направлении восемь кварталов и не встретил ни одного на пути.

    Все дома вровень с землей. Я не знаю, в чем загадка, но в России не бывает дверей без крыльца. Вход в любой магазин, офис или подъезд начинается со ступеней. А рядом пандус с углом в 60 градусов. Почему нельзя вырыть котлован еще на метр глубже? В каких стандартах прописано, что дверь должна быть в метре от земли?

    Это бизнес-центр:

    А это жилой подъезд:

    Если не ясно, в чем вред от бордюров, одолжите инвалидную коляску. Сядьте в нее дома и попробуйте добраться до магазина. Или в обычную коляску уложите ребенка, в нижнюю корзину продукты и вперед через подземный переход, я на вас посмотрю.

    На фотографии ниже видно, что порой действительно случается перепад между тротуаром и зданием. Но оцените, как плавно сделали переход! Высота в 20 сантиметров приходится на длину в 15 метров, угол градусов пять. И как красиво получилось.

    Это тот случай, когда инвалид действительно сможет заехать на коляске, а человек с больным сердцем – подняться без учащения пульса.

    Теперь урны. То ли меня Лебедев укусил, то ли что, но в каждой урне кроется сто важных мелочей, теперь я это понял. В районе, где я жил, урны одинаковы, имеют те же габариты и цвет. Срабатывает визуальное распознавание: чтобы найти урну, достаточно окинуть взором пространство, и где-то обязательно мелькнет оранжевое пятно. В каждой точке видна хотя бы одна урна. Ни разу не было так, чтобы мне понадобилось, а вокруг нет.

    Далее, урна не стоит на земле, а закреплена на столбе на уровне метра. Это позволит увидеть урону на большом расстоянии или в толпе. Отверстие не как у ведра, а меньше и под углом. Здесь тоже умыслел: в такое отверстие не затечет дождь, невозможно заткнуть урну пакетом мусора, как у нас порой делают ленивые уроды.

    В такую урну невозможно что-то кидать, нужно подойти и положить. Это резко снижает желание швырять окурки или бутылки ради понтов. У нас же как: офисный планктон курит на высоком крыльце, где-то внизу ведро. Каждый бросает окурок, это такой вид спорта. Вокруг урны Хи-квадрат-распределение бычков и плевков.

    А попробуй кинь, когда урна на уровне груди. Можно попасть в кого-то и нарваться на неприятности.

    Урны очень аккуратны и чисты, покрашены в оранжевый. В первый раз я принял их за почтовые ящики. Урны никогда не стоят рядом с лавочками, их всегда отделяет метров десять минимум. Пример с автобусной остановкой:

    Странные люди эти немцы, почему не хотят сидеть рядом с урной, как принято у нас? Чтобы справа девушка, а слева мусорное ведро. Удобно же, все рядом.

    В середине дорожных переходов островки безопасности, где пешеход может переждать красный сигнал не рискуя быть сбитым. Островки огорожены бордюрами. Тот редкий случай, когда они действительно нужны.

    Люки преимущественно квадратные. Ливневые решетки спроектированы так, чтобы туда не попало колесо велика или коляски. Направление щелей для воды перпендикулярно движению на этом участке. Если место проходное, делают просто круглые дыры.

    Везде велодорожки и велопарковки, на великах ездят от мала до велика, даже беременные и пенсионеры. У меня нет статистики, но полагаю, это резко снижает дорожный трафик. К примеру, на небольшом пятачке возле магазина припарковано 15 велосипедов. Теперь представим, что каждый приехал на машине. Сколько места понадобиться? В десять раз больше, плюс трафик.

    Дорожные пробки смехотворны по сравнению с нашими. 26 километров по городу от центра до аэропорта – свободно, остановки только на светофорах. В будни после обеда, Карл.

    Многие женщины ходят без лифчика. Не поголовно, конечно, но больше, чем у нас. Часто видно татуировки. Это не фрики с пирсингом, просто небольшие узоры или надписи. Наряжаются кто как хочет, порой провакационно, всем пофиг.

    И это я только про урны и про тратуары. А еще же парки, скверы, дворы и миллион других вещей. Можно днями гулять и выпитывать. В плане городской среды Берлин на высоте.

  • Educational startups

    ,

    Today, educational startups are everywhere. The number of them is growing daily. There hardly might be a week when no new HTML/CSS/Javascript educational site appears on the Internet.

    Sorry if I offend someone, but I don’t believe in remote education through special sites that offer video lessons. More precisely, there might be some effect of course but quite poorer than standard education with people and books.

    About ten years ago, the Internet was full of paid video courses on DVDs. These were home-made lessons made by students where they spoke on basics of PHP+HTML. Today, the startups remind me those DVDs. The only difference is you do not need to order a disk anymore but watch lessons online right after you have paid.

    I’m not running an educational site. I also don’t want to reduce someone’s reputation or business. Let’s just discuss some points that seem important to me.

    Let’s switch to a browser right now and google for “educational startup” phrase. For me, the first link is “Are Education Startups The New Dot Com?” In that article, there is another link titled “Edtech Is The Next Fintech”. Read them, they highlight my concerns.

    Below, there is a bunch of list-like posts titled “N best educational startups”:

    And so on. It was only the two first pages. Let’s check for Angel List then:

    14,615 COMPANIES    2,646 INVESTORS    19,127 FOLLOWERS    2,675 JOBS

    14615 companies. There are about 250 countries in the world. I’ll take rather 200 due to wars or lack of development in some of them. Dividing a number of educational startups on the number of counties gives 14615 / 200 = 73 per country.

    Don’t you think it’s bit more than we need?

    Look, we have about 3 search engines to find anything on the Internet. There are probably 2-5 mobile operators in your country. We’ve got one Wikipedia. But there are 14615 educational companies who want to make money on educational market.

    I really doubt they are about real education

    In fact, there is one goal that a startup tries to reach. It forces users to buy lessons making them think it could really raise their level. Site developers bring challenging factors to manipulate users. These are top-ranked tables, a scale of education decorated as a route with milestones and a crown or a goblet in the end. Any startup brings a chat-room to let users share their success to feel proud.

    I’ve seen several educational sites and must confess they don’t bring any revolutionary ideas. Yes, some of them provide smart environment when you have an IDE into a browser. But the truth is there are only two ways of education that really work. These are people and books.

    Sometimes, people ask me how to improve their experience. Usually they’ve spent some time solving educational tasks but they cannot start a real project. I always answer the same: join a project where professionals work. Being in a team of high-qualified people, you will grow up in a quite short term.

    Reading books helps you to systematize fragments of random knowledge that you’ve got from Twitter, StakOverflow, Wired and so on. Everything that educational sites try to sell you has already been published in books. Really, there are tons of books about PHP, HTML, Java, Python or whatever. You may borrow them for free.

    On the internet, you can by any used book for several dollars. A lesson is usually paid for subscription. In a month, you will lose your access unless you pay again. The book will be yours forever.

    Yes, reading a book is a bit more difficult than watching a video. It’s hard and boring. You even need to interpret code in mind. There is no widgets and chats. Although, it really works.

    A book is really important today since our knowledge is not arranged. We are getting random fragments missing important details. I may compare a book with an asphalt paver that moves slowly but removes any roughness and holes in your mind.

    People and books are the only way to learn

    Recently, I finished reading “Web-development with Clojure” book. Although I thought I knew Clojure pretty well, the book turned into discovery for me. I’ve got plenty of hints and technics that I’m willing to try in the future.

    Education in wide meaning is hard process. Educational startups make you think it has become easy. That is wrong.

    Once I finished reading my first Clojure book, I could solve any primitive task like sorting or finding min/max element in a list using recursion. But it took huge effort to write an HTTP server that manages database connection, renders templates, writes logs, calls Twitter API and so on.

    The reason was all the lessons and tutorials miss some special knowledge about how to manage with complexity. Hot to connect parts of your application together.

    A computer is not a best tool for education. It’s a great tool but also an entertainment center. Ideally, you should turn off all the messagers, close YouTube, Twitter… On your laptop, there a lot of things may interrupt you. Being tired, your brain can always find a pretext to switch on something funny.

    IT-education is not about coding, it also includes negotiations. Sometimes, you might be 100% right but would not be able to deliver your ideas. You may offend your customer with non-suitable manner of speaking. Only people who work close to you may correct that mistakes, not online lessons. By the way, I have never seen a lesson that highlights anything from those mentioned above.

    Of course, I do not blame startups for spreading across the Internet. The reason it happens is a lack of professionals. This is reaction of market. Did you want more programmers? Here you’ve got them. Young people know that IT companies are interested in hiring more people. Oftenly, watching just several videos enough to get PHP/Wordpress job paid in US dollars.

    Universities are not in charge of your further employment. There is a common situation when you’ve just finished the last course but cannot find a job because the industry changes so fast. Nor the government will support you. Education seems to be the last thing they are interested in. Today, my country wages two wars, plays geopolitics and eradicates imported cheese while education level goes down-wise year after year.

    Educational sites might be a hope. But they devalue the real meaning of education.

    The process of self-education is hard. Only you is interested in it, not startups

    OK I’m about to finish and I’ve got a question. I heard, the most powerful language is Lisp. At least Alan Kay said so (a guy who invented OOP). So did Stallman, Dijkstra and other great programmers. Do you know any educational site which offers Lisp lessons?

    I googled for a bit to find any. Nothing on Coursera – the most known lesson hub. Nothing on Netology. At least Hexlet has SICP section where they retell it in Russian (see my note on books).

    No, they won’t teach you the most powerful language. So what is the final goal then? Do we need more Python or Java programmers? We have already got lots of them but we still suffer from weird interfaces and buggy applications. It won’t work in such a way.

    Let me summarize.

    1. I’m not against someone’s business. I even believe some people who watched those videos really made progress in their career. Maybe, they could not find proper books or their brain feels good with such a way of education.
    2. But I’d like to name things properly. Educational startups are the business by themselves. The goal of the business is to make money but not to make you cleverer. Instead, the longer you keep your monthly subscription active, the better a startup develops.
    3. There is definitely overheat on the educational market. It seems to be like a bubble.
    4. Startups tell you the education is easy and funny. It’s not.
    5. People and books help a lot. Online lessons – well, yes but quite less.

  • Weekly links #26

    ,

Страница 55 из 87