Skip to content

Commit 63c5800

Browse files
authored
Extend find-usages to also find anonymous function dependencies
1 parent aa6af7b commit 63c5800

File tree

4 files changed

+151
-21
lines changed

4 files changed

+151
-21
lines changed

CHANGELOG.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,15 @@
22

33
## master (unreleased)
44

5+
## 0.9.0 (2022-01-8)
6+
7+
### Changes
8+
9+
* [#51](https://github.com/clojure-emacs/orchard/issues/51): extend find-usages
10+
* `orchard.xref/fn-deps` now also finds anonymous function dependencies
11+
* added `orchard.xref/fn-deps-class` as a lower level API so you can still get the main functions deps only
12+
* added `orchard.xref/fn-transitive-deps`
13+
514
## 0.8.0 (2021-12-15)
615

716
### Changes

README.md

Lines changed: 21 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -84,12 +84,26 @@ Just add `orchard` as a dependency and start hacking.
8484
Consult the [API documentation](https://cljdoc.org/d/cider/orchard/CURRENT) to get a better idea about the
8585
functionality that's provided.
8686

87-
#### Using `enrich-classpath` for best results
87+
### Using `enrich-classpath` for best results
8888

8989
There are features that Orchard intends to provide (especially, those related to Java interaction) which need to assume a pre-existing initial classpath that already has various desirable items, such as the JDK sources, third-party sources, special jars such as `tools` (for JDK8), a given project's own Java sources... all that is a domain in itself, which is why our [enrich-classpath](https://github.com/clojure-emacs/enrich-classpath) project does it.
9090

9191
For getting the most out of Orchard, it is therefore recommended/necessary to use `enrich-classpath`. Please refer to its installation/usage instructions.
9292

93+
### xref/fn-deps and xref/fn-refs limitations
94+
95+
These functions use a Clojure compiler implementation detail to find references to other function var dependencies.
96+
97+
You can find a more in-depth explanation in this [post](https://lukas-domagala.de/blog/clojure-analysis-and-introspection.html).
98+
99+
The important implications from this are:
100+
101+
* very fast
102+
* functions marked with meta :inline will not be found (inc, +, ...)
103+
* redefining function vars that include lambdas will still return the dependencies of the old plus the new ones
104+
([explanation](https://lukas-domagala.de/blog/clojure-compiler-class-cache.html))
105+
* does not work on AoT compiled functions
106+
93107
## Configuration options
94108

95109
So far, Orchard follows these options, which can be specified as Java system properties
@@ -98,6 +112,12 @@ So far, Orchard follows these options, which can be specified as Java system pro
98112
* `"-Dorchard.initialize-cache.silent=true"` (default: `true`)
99113
* if `false`, the _class info cache_ initialization may print warnings (possibly spurious ones).
100114

115+
## Tests and formatting
116+
117+
To run the CI tasks locally use:
118+
119+
`make test cljfmt kondo eastwood`
120+
101121
## History
102122

103123
Originally [SLIME][] was the most

src/orchard/xref.clj

Lines changed: 98 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -3,30 +3,84 @@
33
references."
44
{:added "0.5"}
55
(:require
6+
[clojure.repl :as repl]
7+
[clojure.set :as set]
8+
[clojure.string :as str]
69
[orchard.query :as q]))
710

8-
(defn- as-val
11+
(defn- to-fn
912
"Convert `thing` to a function value."
1013
[thing]
1114
(cond
1215
(var? thing) (var-get thing)
1316
(symbol? thing) (var-get (find-var thing))
1417
(fn? thing) thing))
1518

19+
(defn- fn-name [^java.lang.Class f]
20+
(-> f .getName repl/demunge symbol))
21+
22+
(defn fn-deps-class
23+
"Returns a set with all the functions invoked by `v`.
24+
`v` can be a function class or a symbol."
25+
{:added "0.9"}
26+
[v]
27+
(let [^java.lang.Class v (if (class? v)
28+
v
29+
(eval v))]
30+
(into #{} (keep (fn [^java.lang.reflect.Field f]
31+
(or (and (identical? clojure.lang.Var (.getType f))
32+
(java.lang.reflect.Modifier/isPublic (.getModifiers f))
33+
(java.lang.reflect.Modifier/isStatic (.getModifiers f))
34+
(-> f .getName (.startsWith "const__"))
35+
(.get f (fn-name v)))
36+
nil))
37+
(.getDeclaredFields v)))))
38+
39+
(def ^:private class-cache
40+
"Reference to Clojures class cache.
41+
This holds of classes compiled by the Clojure compiler,
42+
one class per function and one per repl eval.
43+
This field is package private, so it has to be set to
44+
accessible otherwise an IllegalAccess exception would
45+
be thrown."
46+
(let [classCache* (.getDeclaredField clojure.lang.DynamicClassLoader "classCache")]
47+
(.setAccessible classCache* true)
48+
(.get classCache* clojure.lang.DynamicClassLoader)))
49+
1650
(defn fn-deps
17-
"Returns a set with all the functions invoked by `val`.
18-
`val` can be a function value, a var or a symbol."
51+
"Returns a set with all the functions invoked inside `v` or any contained anonymous functions.
52+
`v` can be a function value, a var or a symbol.
53+
If a function was defined multiple times, old lambda deps will
54+
be returned.
55+
This does not return functions marked with meta :inline like `+`
56+
since they are already compiled away at this point."
1957
{:added "0.5"}
20-
[val]
21-
(let [val (as-val val)]
22-
(set (some->> val class .getDeclaredFields
23-
(keep (fn [^java.lang.reflect.Field f]
24-
(or (and (identical? clojure.lang.Var (.getType f))
25-
(java.lang.reflect.Modifier/isPublic (.getModifiers f))
26-
(java.lang.reflect.Modifier/isStatic (.getModifiers f))
27-
(-> f .getName (.startsWith "const__"))
28-
(.get f val))
29-
nil)))))))
58+
[v]
59+
(when-let [^clojure.lang.AFn v (to-fn v)]
60+
(let [f-class-name (-> v .getClass .getName)]
61+
;; this uses the implementation detail that the clojure compiler always
62+
;; prefixes names of lambdas with the name of its surrounding function class
63+
(into #{} (comp (filter (fn [[k _v]] (clojure.string/includes? k f-class-name)))
64+
(map (fn [[_k value]] (.get ^java.lang.ref.Reference value)))
65+
(mapcat fn-deps-class))
66+
class-cache))))
67+
68+
(defn fn-transitive-deps
69+
"Returns a set with all the functions invoked inside `v` or inside those functions.
70+
`v` can be a function value, a var or a symbol."
71+
{:added "0.9"}
72+
[v]
73+
(let [deps (fn-deps v)]
74+
(loop [checked #{}
75+
to-check (into [] deps)
76+
deps deps]
77+
(cond
78+
(empty? to-check) deps
79+
:else (let [[current & remaining] to-check
80+
new-deps (fn-deps current)]
81+
(recur (conj checked current)
82+
(concat remaining (filter #(contains? deps %) new-deps))
83+
(set/union deps new-deps)))))))
3084

3185
(defn- fn->sym
3286
"Convert a function value `f` to symbol."
@@ -45,8 +99,37 @@
4599
"Find all functions that refer `var`.
46100
`var` can be a function value, a var or a symbol."
47101
{:added "0.5"}
48-
[var]
49-
(let [var (as-var var)
102+
[v]
103+
(let [var (as-var v)
50104
all-vars (q/vars {:ns-query {:project? true} :private? true})
51105
deps-map (zipmap all-vars (map fn-deps all-vars))]
52106
(map first (filter (fn [[_k v]] (contains? v var)) deps-map))))
107+
108+
(comment
109+
;; this can be used to blow up memory, which will clear the class cache of old references
110+
(defn oom []
111+
(try (let [memKiller (java.util.ArrayList.)]
112+
(loop [free 10000000]
113+
(.add memKiller (object-array free))
114+
(.get memKiller 0)
115+
(recur 100000 #_(if (< (Math/abs (.. Runtime (getRuntime) (freeMemory))) Integer/MAX_VALUE)
116+
(Math/abs (.. Runtime (getRuntime) (freeMemory)))
117+
Integer/MAX_VALUE))))
118+
(catch OutOfMemoryError _
119+
(println "freed"))))
120+
121+
(fn-deps #'fn-refs)
122+
(fn-deps #'orchard.xref/fn-deps)
123+
(fn-refs #'orchard.xref/fn->sym)
124+
125+
;; returns all classes in this ns, even repl eval'd
126+
(let [f-class-name "orchard.xref" #_(-> orchard.xref/fn-deps .getClass .getName)
127+
classes (into #{} (comp (filter (fn [[k _v]] (clojure.string/includes? k f-class-name)))
128+
(map (fn [[_k v]] (.get ^java.lang.ref.Reference v))))
129+
class-cache)]
130+
classes)
131+
132+
(oom)
133+
(def vars (q/vars {:ns-query {:project? true} :private? true}))
134+
135+
(map fn-deps vars))

test/orchard/xref_test.clj

Lines changed: 23 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -3,22 +3,28 @@
33
[clojure.test :refer [deftest is testing]]
44
[orchard.xref :as xref]))
55

6+
(defn- fn-transitive-dep [a b]
7+
(* a b))
8+
9+
(defn- fn-dep [a b]
10+
(fn-transitive-dep a b))
11+
612
(defn- dummy-fn [_x]
7-
(map #(* % 2) (filter even? (range 1 10))))
13+
(map #(fn-dep % 2) (filter even? (range 1 10))))
814

915
(deftest fn-deps-test
1016
(testing "with a fn value"
1117
(is (= (xref/fn-deps dummy-fn)
1218
#{#'clojure.core/map #'clojure.core/filter
13-
#'clojure.core/even? #'clojure.core/range})))
19+
#'clojure.core/even? #'clojure.core/range #'orchard.xref-test/fn-dep})))
1420
(testing "with a var"
1521
(is (= (xref/fn-deps #'dummy-fn)
1622
#{#'clojure.core/map #'clojure.core/filter
17-
#'clojure.core/even? #'clojure.core/range})))
23+
#'clojure.core/even? #'clojure.core/range #'orchard.xref-test/fn-dep})))
1824
(testing "with a symbol"
1925
(is (= (xref/fn-deps 'orchard.xref-test/dummy-fn)
2026
#{#'clojure.core/map #'clojure.core/filter
21-
#'clojure.core/even? #'clojure.core/range}))))
27+
#'clojure.core/even? #'clojure.core/range #'orchard.xref-test/fn-dep}))))
2228

2329
;; The mere presence of this var can reproduce a certain issue. See:
2430
;; https://github.com/clojure-emacs/orchard/issues/135#issuecomment-939731698
@@ -38,4 +44,16 @@
3844
(is (contains? (into #{} (xref/fn-refs #'map)) #'orchard.xref-test/dummy-fn)))
3945
(testing "with a symbol"
4046
(is (= (xref/fn-refs 'orchard.xref-test/dummy-fn) '()))
41-
(is (contains? (into #{} (xref/fn-refs #'map)) #'orchard.xref-test/dummy-fn))))
47+
(is (contains? (into #{} (xref/fn-refs #'map)) #'orchard.xref-test/dummy-fn)))
48+
(testing "that usage from inside an anonymous function is found"
49+
(is (contains? (into #{} (xref/fn-refs #'fn-dep)) #'orchard.xref-test/dummy-fn))))
50+
51+
(deftest fn-transitive-deps-test
52+
(testing "basics"
53+
(let [expected #{#'orchard.xref-test/fn-deps-test #'orchard.xref-test/fn-dep #'clojure.core/even?
54+
#'clojure.core/filter #'orchard.xref-test/fn-transitive-dep #'clojure.core/map
55+
#'clojure.test/test-var #'clojure.core/range}]
56+
(is (contains? expected #'orchard.xref-test/fn-transitive-dep)
57+
"Specifically includes `#'fn-transitive-dep`, which is a transitive dep of `#'dummy-fn` (via `#'fn-dep`)")
58+
(is (= expected
59+
(xref/fn-transitive-deps dummy-fn))))))

0 commit comments

Comments
 (0)