Use Server-Sent Events in the Couch DB adapter

[akhenry]

Server Sent Events provide a cleaner interface for receiving change events from the Couch Server than handling chunked HTTP responses.

[scottbell]

@akhenry I'm a bit concerned with this aspect of Server-Sent events:
https://developer.mozilla.org/en-US/docs/Web/API/EventSource

Warning: When not used over HTTP/2, SSE suffers from a limitation to the maximum number of open connections, which can be specially painful when opening various tabs as the limit is per browser and set to a very low number (6). The issue has been marked as "Won't fix" in Chrome and Firefox. This limit is per browser + domain, so that means that you can open 6 SSE connections across all of the tabs to www.example1.com and another 6 SSE connections to www.example2.com. (from Stackoverflow). When using HTTP/2, the maximum number of simultaneous HTTP streams is negotiated between the server and the client (defaults to 100).

Namely, if a user has 6 tabs open to OpenMCT, we may start failing to persist objects on the 7th tab. Is this right?

[scottbell]

I can look at possible multiplexing across tabs (maybe using ServiceWorkers?), or feature-flagging SSE too.

[akhenry]

@scottbell This issue affects all HTTP/1 requests unfortunately. Of course the problem is exacerbated with both SSEs and chunked responses (our current strategy) because they hold the connection open indefinitely. We are solving this right now by multiplexing through a shared worker. So long as we continue to use a shared worker with SSEs we're fine.

On a side note, Safari does not support shared workers, because they inexplicably removed support for them years ago. There are some recent signs that they may be re-implementing support though. So right now we do feature detection for SharedWorkers and just fall back on a regular connection if SharedWorkers are not available. Our only use case for Safari is mobile support in iOS, which is a requirement. Hopefully Safari will get shared workers soon, and if it doesn't we will have to figure something out for iOS down the line.

[scottbell]

We've run into similar issues with shared workers and Safari on other projects, so I feel your pain. One thing to think about though with iOS, users typically do less tabbed browsing, so maybe a straight SSE connection would be ok.

In any event, I will start the SSE work using shared workers! Thanks for the insights

[scottbell]

@akhenry Between 1.7.7 and 1.7.8, when using the CouchDB persistence plugin, this line:
https://github.com/nasa/openmct/blob/master/platform/commonUI/edit/src/creation/CreateAction.js#L81
is failing as object.getCapability('context') returns null. This causes some exciting behavior in the UI:

I'm digging into what changed trying to figure out what's causing the context capability to be null. LocalStorage is working fine in both versions.

[scottbell]

I think the source of the bug is before the navigateAndEdit. The Missing: appears well before that line is evaluated

[scottbell]

Yeah, looks like the underlying model is missing. I wonder if it's a timing issue. Namely, we're trying to persist before the model is ready

[scottbell]

Ah, narrowed it down a bit. Undoing this PR:
#4195
seems to resolve the issue. Now to find out what about it broke Couch!

[scottbell]

Ah, rumor has it @shefalijoshi may have a fix. I'll chat with her.

[scottbell]

Found @shefalijoshi's fix:
49588bf

[scottbell]

Though I've resolved the issue of saving objects to CouchDB through @shefalijoshi's fix, it looks like the synchronization is still broken. For example, creating an object of SYNCHRONIZED_OBJECT_TYPES (like a notebook) causes the shared worker to engage, but then immediately fails:

Editor.js?3ce0:82 Uncaught (in promise) TypeError: Cannot read properties of null (reading 'cancel')
    at eval (Editor.js?3ce0:82)
    at new Promise (<anonymous>)
    at Editor.cancel (Editor.js?3ce0:80)
    at onCancel (CreateAction.js?95e8:71)
    at processQueue (angular.js?21b1:18075)
    at eval (angular.js?21b1:18123)
    at Scope.$digest (angular.js?21b1:19242)
    at eval (angular.js?21b1:19562)
    at TaskTracker.completeTask (angular.js?21b1:21403)
    at eval (angular.js?21b1:6879)

This is because the active transaction is null:

const transaction = this.openmct.objects.getActiveTransaction();
            transaction.cancel()

which causes the cancel invocation to crash. I'll investigate

[scottbell]

This appears to be a timing issue. This:
https://github.com/nasa/openmct/blob/master/src/api/Editor.js#L69
this.openmct.objects.endTransaction();
Is being called before this:
https://github.com/nasa/openmct/blob/master/src/api/Editor.js#L81
const transaction = this.openmct.objects.getActiveTransaction();
thus the ActiveTransaction is null.
It's not clear to me why we're invoking cancel in the first place though.

[scottbell]

I think the cancel is happening because the SaveAsAction is failing due to:

TypeError: Cannot read properties of undefined (reading 'filter')
    at eval (CouchObjectProvider.js?a10c:373)
    at EventEmitter.eval (ObjectAPI.js?4467:495)
    at EventEmitter.emit (index.js?ba10:115)
    at MutableDomainObject.$destroy (MutableDomainObject.js?3566:123)
    at ObjectAPI.destroyMutable (ObjectAPI.js?4467:286)
    at ObjectAPI.__webpack_exports__.default.openmct.objects.save (monkeyPatchObjectAPIForNotebooks.js?87fc:23)
    at async Promise.all (:8080/index 0)

[scottbell]

Still looks like an interplay between transactions and what the persistence plugin is expecting. In SaveAsAction, we call persist:
https://github.com/nasa/openmct/blob/master/platform/commonUI/edit/src/actions/SaveAsAction.js#L136
which invokes a patch that destroys the mutable object:
https://github.com/nasa/openmct/blob/master/src/plugins/notebook/monkeyPatchObjectAPIForNotebooks.js#L23
which in turn calls unobserve for the first time:
https://github.com/nasa/openmct/blob/master/src/plugins/persistence/couch/CouchObjectProvider.js#L373
this.observers[keyString] = this.observers[keyString].filter(observer => observer !== callback);
which replaces the this.observers[keyString] with null if observer === callback

Later, the transaction is committed, and this call save, which invokes the patch a second time that destroys the mutable object:
https://github.com/nasa/openmct/blob/master/src/plugins/notebook/monkeyPatchObjectAPIForNotebooks.js#L23
which signals again to the CouchObjectProvider to unobserve, but the this.observers is already null, and thus the raised exception when trying to filter on it.

[scottbell]

Adding a null check seems to fix it:

return () => {
            if (this.observers[keyString]) {
                this.observers[keyString] = this.observers[keyString].filter(observer => observer !== callback);
                if (this.observers[keyString].length === 0) {
                    delete this.observers[keyString];
                    if (Object.keys(this.observers).length === 0 && this.isObservingObjectChanges()) {
                        this.stopObservingObjectChanges();
                    }
                }
            }
        };

@akhenry @shefalijoshi I'm a bit lost on the observer interaction, but would the above be ok?

[scottbell]

Split the Notebook Object issue to:
#4422

[scottbell]

A PR for this issue has been opened here:
#4427

[charlesh88]

Testathon 12-13-21 - no testing notes.

[khalidadil]

@scottbell Is there a good way to test this?

[unlikelyzero]

Testing Notes

Each of the following must be performed with Safari and also with Chrome (not at the same time).

1. Open multiple tabs and verify that notebook entries are modified between users. Without Refresh.
2. Modify the name of a clock object and verify that change will take effect on other tabs. With Refresh.

[khalidadil]

I'm seeing an issue where opening notebook another tab results in the user not being able to create an entry on the second tab.

Steps to reproduce:

1. Open a notebook in tab 1. Create an entry. The entry should be created successfully.
2. Open the same notebook in tab 2 in the same browser. Create an entry. The entry input appears, but then the entry disappears.

@michaelrogers noted that you can navigate away from the notebook and back to it in the same tab to get in this same state.

Screen.Recording.2022-01-06.at.5.02.42.PM.mov

[shefalijoshi]

Tried to modify a notebook entry in different tabs and the entries did not update consistently. I think this could have something to do with default notebooks not working as expected. I suggest waiting for #4469 to be merged before testing this again.

[scottbell]

Post #4469 being merged, it looks like it's working in 1.8.3 (at least on my local install):

Screen.Recording.2022-01-07.at.12.54.40.PM.mov

[unlikelyzero]

Re-opening. There are some scenarios which are not working and need additional debugging.

Note this may have to do with active / inactive tab

[akhenry]

Closed in favor of #4737. After investigation it has been determined that notebook issues are unrelated to Server Sent Events.