Pixellated Portrait of a ginger with a beard.

Danger
Russ
Notes

home

about

mastodon

github

2022-11-21

Published: Dec 05, 2022Last modified: Apr 24, 2023
#clawe#cli#cljs#clojure#emacs#goals#godot#kaocha#os#techdebt#tools
Word count: 1275

i ought to learn how to do this - emacs font locking

#emacs

maybe start some weekly emacs feats/builds? or a week/month sprint on it?

what's this fancy new thing? - godot movie maker mode

#godot

Transforming Data With Malli and Meander - Metosin

#clojure

[Transforming Data With Malli and Meander - Metosin](https://www.metosin.fi/blog/transforming-data-with-malli-and-meander/)

malli + meander

Welcome to osquery

#os#clawe

maybe useful for ralphie / clawe

[Welcome to osquery - osquery](https://osquery.readthedocs.io/en/stable/)

get open ports, processes, etc?

plus yogthos bb example

make a monster Cli list ?

#tools#cli

Godot weather app?!

#godot

goals

#goals

> DONE inbox zero

> public patreon post/update: focus + pomodoro component

> DONE clawe tests running in CI

>> DONE prevent lichess requests

>> DONE disable integration window-manager tests

>> DONE run clj unit tests

>> DONE run bb unit tests

> DONE run cljs unit tests

> clips from existing streams

> TODO games journal indexing continued

> TODO org-blog internal publish/export app

> TODO advent of code prep

godot approach?

trying to make cljs testing work

#clojure#cljs#kaocha

Some raw notes taken while trying to get a reasonable cljs test runner in place in clawe.

Trying to capture a bunch - i'll improve/refactor these into a better approach once things are running.

> y so complex?

is it really this complicated to run cljs tests for a web app?

A high-level lesson I'm learning that shadow-cljs has protected me from tons of cljs complexity. I've never really needed to know what these cljs compiler options are, or where the node-module integration magic happens. (vs trying to get tests running, i'm seeing errors related to these all over).

> kaocha-cljs kind of a pita

seems more complicated than necessary? And they point to a harder-to-configure kaocha-cljs2?

I expected to be able to write+config completely within deps.edn or bb.edn, but it wants you to create a ./bin/kaocha and a ./tests.edn. I think it's possible to do all this config in deps.edn, but it feels discouraged.

Tho, it is certainly supported: https://github.com/lambdaisland/kaocha/blob/main/doc/02_installing.md#alternative-method-exec-fn It's not totally clear how tests.edn vs :exec-args combine... and after more digging, maybe deps.edn isn't a good place for the test config, because it will be duplicated across aliases.

  :doctor-test-clj  {:extra-deps {lambdaisland/kaocha          {:mvn/version "1.71.1119"}
                                  com.lambdaisland/kaocha-cljs {:mvn/version "1.4.130"}}
                     :exec-fn    kaocha.runner/exec-fn
                     :exec-args  {}}
  :doctor-test-cljs {:extra-deps {lambdaisland/kaocha          {:mvn/version "1.71.1119"}
                                  com.lambdaisland/kaocha-cljs {:mvn/version "1.4.130"}}
                     :exec-fn    kaocha.runner/exec-fn
                     :exec-args  {}}

  :doctor-test-watch {:extra-deps {lambdaisland/kaocha          {:mvn/version "1.71.1119"}
                                   com.lambdaisland/kaocha-cljs {:mvn/version "1.4.130"}}
                      :exec-fn    kaocha.runner/exec-fn
                      :exec-args  {:watch?     true
                                         :skip-meta  :slow
                                   :fail-fast? true}}

Probably a job for a bb.edn to dry up the tasks into some re-usable variables.

Reducing the number of places things get configured is important to avoid

nonsense, like the paths issue i ran into below (b/c the config was buried in a

third *.edn file.)

---

kaocha loads all my clerk notebooks, which is a problem. Hopefully i'll be able to ignore those - in my case this means we're fetching against the lichess api every test run unless i disable it.

Loading all namespaces is pretty normal for clojure tools, i suppose. Maybe my own app does this at startup, and I'm just not aware b/c of all my deprecated namespace style warnings/noise. Have really got to get plasma off these old uix deps.

---

Loading the clerk notebooks was my fault - i'd included "src" in the :test-paths

in tests.edn

;; tests.edn
#kaocha/v1
{:tests [{:id         :unit
          :test-paths ["test" "src"]}]} ;; <-- bad!

vs:

;; tests.edn
#kaocha/v1
{:tests [{:id           :unit
          :source-paths ["src"]
          :test-paths   ["test"]}]} ;; <-- ok.

and apparently these are set by default, so don't need to specified at all

>> what if i want clj and cljs tests?

In trying to run the jvm/clj side, this 'just-data' config will crash with a

'unable to resolve spec: :kaocha.type/cljs' error. :(

;; tests.edn
#kaocha/v1
{:tests [{:id :unit}
         {:id            :unit-cljs
          :type          :kaocha.type/cljs
          :cljs/repl-env cljs.repl.browser/repl-env
          :source-paths  ["src"]
          :test-paths    ["test"]
          ;; :cljs/timeout 10000 ; 10 seconds, the default
          }]}

deps.edn alias:

  :doctor-test-clj  {:extra-deps {lambdaisland/kaocha {:mvn/version "1.71.1119"}}
                     :exec-fn    kaocha.runner/exec-fn
                     :exec-args  {}}

Adding the kaocha-cljs alias then runs the cljs tests. (well, it crashes b/c one of my node-dep-based-namespaces can't be resolved). I was hoping to run the backend tests? What's up with that? Not sure how it picks which test suite to run, i thought I read :unit was the default.

I'm sure this big bundle of test-runner can be coaxed into behaving properly. Back to reading the docs!

---

Turns out this was b/c it runs all test suites by default - my bad again! Not sure who to indicate that i only want to run one suite without the magic bin/kaocha file :( I'd like to write a bb.edn task instead, but i'm having trouble making it work - coding in the dark a bit - can't even get the help text to print.

>> bb-based cli rewrite would be a win

Right now i'm waiting 26s to see if this prints the help text or not

clj -X:full-stack-deps:doctor-test-clj :test-help true

Mostly my own fault for not knowing tools.deps command line well enough.

And i'm avoiding creating a kaocha bin file just to pass args to it.... tho at this point i have to give up, it seems to be the only way to pass args.

>> fine! I finally folded and created ./bin/kaocha

#!/usr/bin/env sh

clojure -M:full-stack-deps:kaocha "$@"

and deps.edn

  :kaocha {:extra-deps
           {lambdaisland/kaocha          {:mvn/version "1.71.1119"}
            com.lambdaisland/kaocha-cljs {:mvn/version "1.4.130"}}
           :main-opts ["-m" "kaocha.runner"]}

and I can no see the help text.

For anyone looking:

USAGE:

bin/kaocha [OPTIONS]... [TEST-SUITE]...

  -c, --config-file FILE     tests.edn  Config file to read.
      --print-config                    Print out the fully merged and normalized config, then exit.
      --print-test-plan                 Load tests, build up a test plan, then print out the test plan and exit.
      --print-result                    Print the test result map as returned by the Kaocha API.
      --[no-]fail-fast                  Stop testing after the first failure.
      --[no-]color                      Enable/disable ANSI color codes in output. Defaults to true.
      --[no-]watch                      Watch filesystem for changes and re-run tests.
      --reporter SYMBOL                 Change the test reporter, can be specified multiple times.
      --diff-style STYLE                The style of diff to print on failing tests, either :none or :deep
      --plugin KEYWORD                  Load the given plugin.
      --profile KEYWORD                 Configuration profile. Defaults to :default or :ci.
      --version                         Print version information and quit.
      --help                            Display this help message.
  -H, --test-help                       Display this help message.
      --[no-]randomize                  Run test namespaces and vars in random order.
      --seed SEED                       Provide a seed to determine the random order of tests.
      --skip SYM                        Skip tests with this ID and their children.
      --focus SYM                       Only run this test, skip others.
      --skip-meta SYM                   Skip tests where this metadata key is truthy.
      --focus-meta SYM                  Only run tests where this metadata key is truthy.
      --[no-]capture-output             Capture output during tests.

Options may be repeated multiple times for a logical OR effect.

and I can run the clj/cljs tests independently:

./bin/kaocha unit
./bin/kaocha unit-cljs

Note that the cljs files still get loaded in the `unit` test case.

>> skip-meta and focus-meta work great

I have two clj tests running - some that will work in CI (hopefully) and others that require a working window manager. Maybe i'll be able to run those in an awesome-wm box ?? not worrying about that for now.

On to the cljs debugging!

>> no such namespace for some-npm-dep

not sure what configuration is required for node deps to just-work - it's not mentioned in the readme, so now we're searching issue history for what is definitely a cljs-testing use-case.

maybe this is just completely unsupported? https://github.com/lambdaisland/kaocha-cljs/issues/31

having seen that, I'm curious what they meant by just using shadow-cljs's testing setup...

looks like shadow provides some testing capabilities out of the box: https://shadow-cljs.github.io/docs/UsersGuide.html#_testing and kaocha-cljs does not support integrating with shadow: https://github.com/lambdaisland/kaocha-cljs/issues/2 - thus the cljs2 rewrite, it seems

>> every attempt is a 20s endeavor of opts

and the command line tool doesn't correlate perfectly to the opts... so we're configging test.edn often, with opts that we don't want to live forever...

being in bb.edn or having these resonably cli supported would be better.

current tests.edn:

;; tests.edn
#kaocha/v1
{:tests [{:id        :unit
          :skip-meta [:integration]}
         {:id         :integration
          :focus-meta [:integration]}
         {:id                    :unit-cljs
          :type                  :kaocha.type/cljs
          :test-paths            ["test"]
          :cljs/repl-env         cljs.repl.browser/repl-env
          ;; :cljs/repl-env     cljs.repl.node/repl-env
          ;; :cljs/timeout 10000 ; 10 seconds, the default
          :cljs/compiler-options {:npm-deps true
                                  :verbose  true
                                  ;; :foreign-libs
                                  ;; [{:file           "dist/index_bundle.js"
                                  ;;   :provides       ["react" "sentry"]
                                  ;;   :global-exports {react  React
                                  ;;                    sentry Sentry}}]
                                  }}]

 :reporter        [kaocha.report/debug]
 :bindings        {kaocha.type.cljs/*debug* true}
 :capture-output? false}

>> cljs maintainers read as burned out

> turns out shadow provides test runner instructions that just-work

 :dev-http {8021 "out/test"}
 :builds
 {:doctor-test
  {:target   :browser-test
   :test-dir "out/test"}
  :karma
  {:target    :karma
   :output-to "target/ci.js"}}

you can open a browser to 8021 to see a basic test runner layout, or run karma on the command line to run your tests in CI-style.

shadow-cljs compile karma && npx karma start --single-run

and, i'm hitting precisely the same errors as I saw from the olical test runner hours ago, so maybe that path was actually the best way to go.

> olical/cljs-test-runner actually runs my tests out of the box

this is the ux i thought i'd get from kaocha - nice to know this exists

https://github.com/Olical/cljs-test-runner

Tho, this ran on node (not in a browser) - maybe that's why the node-modules just-worked, and that's something i had configured in my first attempt kaocha-cljs.

in my case, this didn't work right away - for some reason my tick readers were being converted into java calls, which then didn't work at all from node.

> maybe etaoin works - feels newer and maybe like it has a modern UX

tho it's webdriver, not really a test runner, so not really the same thing

https://github.com/clj-commons/etaoin

tech debt wishlist

#techdebt

> org item deduplication/clean up-on-edit

it would be excellent to have a datalog db that is always-up-to-date with the latest items from org.

the trouble is items without uuids - arbitrary edits can clear them, which is a

problem if we want to attach metadata to them

one answer: attach a uuid whenever we attach metadata

that much is fine, but the more pressing problem: we need to invalidate/clear items when a

file is saved and items have been deleted/edited

> socket reconnection

plasma, clerk, etc

sockets that don't reconnect don't survive edits/changes/restarts from the backend

home

about

mastodon

github