Clone this repo:


  1. b7abbf4 Merge pull request #1 from akihikodaki/master by Jason Riesa · 3 weeks ago master
  2. 3bfd864 Add an instruction to build the demo in the installation instruction by Akihiko Odaki · 3 weeks ago
  3. b8cc011 Correct the path in the installation instruction by Akihiko Odaki · 3 weeks ago
  4. ae02d6b static_cast-ing a few chars to unsigned chars because on some platforms by default char is signed, on others it's unsigned by Anton Bakalov · 5 months ago
  5. c03368e Updating the feature spec parser by Anton Bakalov · 6 months ago

Compact Language Detector v3 (CLD3)


CLD3 is a neural network model for language identification. This package contains the inference code and a trained model. The inference code extracts character ngrams from the input text and computes the fraction of times each of them appears. For example, as shown in the figure below, if the input text is “banana”, then one of the extracted trigrams is “ana” and the corresponding fraction is 2/4. The ngrams are hashed down to an id within a small range, and each id is represented by a dense embedding vector estimated during training.

The model averages the embeddings corresponding to each ngram type according to the fractions, and the averaged embeddings are concatenated to produce the embedding layer. The remaining components of the network are a hidden (Rectified linear) layer and a softmax layer.

To get a language prediction for the input text, we simply perform a forward pass through the network.



CLD3 is designed to run in the Chrome browser, so it relies on code in Chromium. The steps for building and running the demo of the language detection model are:

  • check out the Chromium repository.
  • copy the code to //third_party/cld_3
  • Uncomment language_identifier_main executable in src/
  • build and run the model using the commands:
gn gen out/Default
ninja -C out/Default third_party/cld_3/src/src:language_identifier_main


To ask questions or report issues please contact


Original authors of the code in this package include (in alphabetical order):

  • Alex Salcianu
  • Andy Golding
  • Anton Bakalov
  • Chris Alberti
  • Daniel Andor
  • David Weiss
  • Emily Pitler
  • Greg Coppola
  • Jason Riesa
  • Kuzman Ganchev
  • Michael Ringgaard
  • Nan Hua
  • Ryan McDonald
  • Slav Petrov
  • Stefan Istrate
  • Terry Koo