Add support for MSC3575: Sliding Sync (#2242)

* sliding sync: add client function and add stub sliding-sync.ts

Mostly c/p from sync.ts. Define interfaces for MSC3575 sliding
sync types. Complete WIP!

* Add core sliding sync classes

* Add integration tests for sliding sync api basics

* gut unused code; add more types

* Use SlidingSync in MatrixClient; stub functions for Sync

Enough to make ele-web actually load okay with 0 rooms.

* Start feeding through room data to the client

* Bugfixes so it sorta ish works

* Refactor the public API for sliding sync

Still needs some work but it's a start.

* Use EventEmitter for callbacks. Add ability to adjust lists and listen for list updates.

- Have atomic getList/setList operations on SlidingSync to update windows etc
- Add a list callback which is invoked with the list indicies and joined count.

* Add stub tests; add listenUntil to make tests easier to read

* No need to resend now

* Add more sliding sync tests; add new setListRanges function

* build tests upon one another to reduce boilerplate and c/p

* More thorough sliding sync tests

* Dependency inject SlidingSync in Client opts when calling startClient()

* Linting

* Fix crash when opts is undefined

* Fix up docs to make CI happy

* Remove all listeners when stop()d to allow for GC

* Add support for extensions

* Add ExtensionE2EE automatically if opts.crypto is present

* Add ExtensionToDevice automatically

* Bugfixes for to_device message processing

* default events to []

* bugfix: don't tightloop when the server is down

Caused by not detecting abort() correctly

* Return null for bad index positions

* Add getListData to get the initial calculated list response

* Add is_tombstoned

* More comments

* Add support for account data extension; rejig extension interface

* Handle invite_state

* Feed through prev_batch tokens

* Linting

* Fix tests

* Linting

* Iterate PR

* Iterate tests and remove unused code

* Update matrix-mock-request

* Make tests happier

* Remove DEBUG/debuglog and use logger.debug

* Update the API to the latest MSC; fixup tests

* Use undefined not null to make it work with the latest changes

* Don't recreate rooms when initial: true

* Add defensive code when unsigned.transaction_id is missing

We can still pair up events by looking at the event_id. We need
to do this in Sliding Sync because the proxy has limitations that
means it cannot guarantee it will always incude a transaction_id
in unsigned. The main reason why is due to the following race condition:
 - A and B are in a DM room.
 - Both are using the proxy.
 - A says "hello".
 - B's sync stream gets "hello" on the proxy. At this point the proxy
   knows it needs to deliver it to A. It does so, but this event has
   no transaction_id as it came down B's sync stream, not A's.
 - If instead, A's sync stream gets "hello" on the proxy, the proxy
   will deliver this message with the transaction_id correctly set.

There are no guarantees that A's sync stream will get the event in a
timely manner, hence the decision to just deliver the events as soon
as the proxy gets the event. This will not be an issue for native
Sliding Sync implementations; this is just a proxy issue.

* Linting

* Add additional sliding sync tests

* Begin adding SlidingSyncSdk tests

* Linting

* Add more sliding sync sdk tests

* Prep work for extension tests

* Linting

* Add account data extension tests

* add to-device tests

* Add E2EE extension tests

* Code smell fixes and extra tests

* Add test for no-txn-id local echo

* Add tests for resolveProfilesToInvites

* Add tests for moving entries down as well as up the list

* Remove conn-management.ts

* Actually verify the event was removed from the txn map

* Handle the case when /sync returns before /send without a txn_id

And ensure all the tests actually test the right things.

* Linting

Co-authored-by: Michael Telatynski <[email protected]>

Author: kegsay

package.json

@@ -102,7 +102,7 @@
     "jest-localstorage-mock": "^2.4.6",
     "jest-sonar-reporter": "^2.0.0",
     "jsdoc": "^3.6.6",
-    "matrix-mock-request": "^2.0.1",
+    "matrix-mock-request": "^2.1.0",
     "rimraf": "^3.0.2",
     "terser": "^5.5.1",
     "tsify": "^5.0.2",