Skip to content

Save actions #761

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 104 commits into from
Oct 9, 2014
Merged

Save actions #761

merged 104 commits into from
Oct 9, 2014

Conversation

kiritsuku
Copy link
Member

This is not yet completely finished but because I already used it in production and it worked great so far I want to give you the possibility to play with it around as early as possible. In the following a short overview about what is included in this PR.

Features included:

  • Java save actions do no longer work for Scala editors
  • Instead there are even better Scala save actions. They look like this:
    sa-code
  • A preference page in Scala → Editor → Save actions where the save actions can be enabled/disabled:
    sa-pref
  • A timeout for save actions (can be configured in the preferences). This timeout keeps the editor responsive but can't abort the computation of the save action (in the worst case it means that a non returning save action will run until termination of Eclipse in a background thread)
  • Document driven (fast) and compiler driven (slow) save actions. The former usually operates in ~1-10ms, the latter more in ~50-timeout ms.

Design overview:

  • Package org.scalaide.extensions contains new API+save actions
  • Package org.scalaide.core.text also contains new API
  • Package org.scalaide.core.internal.{text,extensions} contains implementations of the API
  • Package org.scalaide.ui.internal.editor contains additions needed for save actions.
  • A new save action needs to be a trait X extends SaveAction with {CompilerSupport,DocumentSupport}
  • The save action needs to be enabled in SaveActionExtensions.saveActionSettings and in one of SaveActionExtensions#{applyDocumentExtensions,applyCompilerExtensions}

Drawbacks/TODOs:

  • The introduction of ScalaDocumentProvider broke several things
    • Implicit highlighting (SemanticHighlightingReconciliation)
    • Semantic highlighting extensions (participants)
    • Toggle breakpoint (in ScalaToggleBreakpointAdapter)
    • BaseSemanticAction.refresh
    • All other things, which call JavaPlugin.getDefault.getWorkingCopyManager.getWorkingCopy
    • Quick fixes are not shown anymore
    • Diff viewer (for Scala files)
    • EditorUtility.openInEditor creates InternalClassFileEditorInput, whose getAdapter method is broken → it needs to a ScalaClassFile.
  • compiler accesses are not yet wrapped in askOption calls
  • API is not yet finished
  • No tests for the save actions yet
  • Save action preference page preview
  • Resource leak when a save action never returns. I see possibilities to "reactify" the save action API because most functionality is hidden to users. But it is not yet implemented.
  • Bugs in AddMissingOverride:
  • Don't create save action when it is not enabled
  • Bugs in AddReturnTypeToPublicSymbols:
  • Update docs: Documentation for semantic annotation extension to Scala IDE docs#99 (comment)
  • missing features

@ghprb-bot
Copy link

Test PASSed.
Refer to this link for build results: https://jenkins.scala-ide.org:8496/jenkins/job/ghprb-scala-ide-validator/955/

@ghprb-bot
Copy link

Test FAILed.
Refer to this link for build results: https://jenkins.scala-ide.org:8496/jenkins/job/ghprb-scala-ide-validator/964/

@kiritsuku
Copy link
Member Author

It fails with Autolaunch error: X11 initialization failed., which is weird since I can't remember introducing tests that rely on GUI code. Could someone please look if this is the fault of my PR or something else?

@ghprb-bot
Copy link

Test FAILed.
Refer to this link for build results: https://jenkins.scala-ide.org:8496/jenkins/job/ghprb-scala-ide-validator/965/

/**
* Applies a transformation to the tree of the saved document.
*/
final def transformFile(trans: Transformation[Tree, Tree]): Seq[Change] =
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code looks very familiar to me, could we change scala refactoring so you don't have to copy paste as much code?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought about it but didn't come up with a good solution. I have my own Change objects, that is why I couldn't reuse the code from scala-refactoring. I want to play around with some cancellation conditions anyway, where own code is useful, but I can't say at the moment to what extend it makes sense to move the changes to scala-refactoring.

@misto
Copy link
Member

misto commented Aug 14, 2014

I took a look at the code, but I'm not sure what causes the UI dependency and the tests to fail. Could it be the ScalaDocumentProvider in the ScalaPlugin? But that's just a guess..

@kiritsuku
Copy link
Member Author

That could be it, the ScalaDocumentProvider relies on UI classes. In this case I have to move it to its instantiation to a different place.

@kiritsuku
Copy link
Member Author

Now the error message makes totally sense to me, ScalaDocumentProvider is indeed the problem:

[org.eclipse.equinox.weaving.aspectj] info weaving bundle 'org.eclipse.jdt.ui'
[org.eclipse.jdt.ui] error can't determine modifiers of missing type org.eclipse.pde.internal.ui.wizards.imports.PluginImportHelper
when weaving type org.eclipse.jdt.internal.ui.javaeditor.CompilationUnitDocumentProvider
when weaving classes 
when weaving

@skyluc
Copy link
Member

skyluc commented Aug 14, 2014

Yes, it is a problem when trying to load 'ScalaDocumentProvider' to use in 'ScalaPlugin'.

java.lang.UnsatisfiedLinkError: Could not load SWT library. Reasons:
  /srv/xdata2/dev/scala-ide/scala-ide/org.scala-ide.sdt.core.tests/target/work/configuration/org.eclipse.osgi/bundles/123/1/.cp/libswt-pi-gtk-4335.so: libgtk-x11-2.0.so.0: cannot open shared object file: No such file or directory
  no swt-pi-gtk in java.library.path
  Can't load library: /home/luc/.swt/lib/linux/x86_64/libswt-pi-gtk-4335.so
  Can't load library: /home/luc/.swt/lib/linux/x86_64/libswt-pi-gtk.so
  /home/luc/.swt/lib/linux/x86_64/libswt-pi-gtk-4335.so: libgtk-x11-2.0.so.0: cannot open shared object file: No such file or directory

  at org.eclipse.swt.internal.Library.loadLibrary(Library.java:331)
  ....
  at org.eclipse.swt.widgets.Display.<clinit>(Display.java:133)
  at org.eclipse.jface.preference.PreferenceConverter.<clinit>(PreferenceConverter.java:79)
  at org.eclipse.jdt.ui.PreferenceConstants.initializeDefaultValues(PreferenceConstants.java:3739)
  at org.eclipse.jdt.internal.ui.JavaUIPreferenceInitializer.initializeDefaultPreferences(JavaUIPreferenceInitializer.java:38)
  ....
  at org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.loadClass(DefaultClassLoader.java:107)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
  at org.scalaide.core.ScalaPlugin.<init>(ScalaPlugin.scala:176)
  ....

I can give you the full stack traces if you want.

@kiritsuku
Copy link
Member Author

After making the documentProvider field a lazy val, the problem is solved. That is why there are so many lazy vals in there.

with SaveActionExtensions {

/** We need to access private values of the super class */
private val ra = ReflectAccess[CompilationUnitDocumentProvider](this)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think reflexion is require. It is used for 2 attributes (fIsAboutToSave and fSavePolicy) which are only used in method you already override, or which would be easy to override (setSavePolicy and saveDocumentContent).
You could have local copies of this values.

@skyluc
Copy link
Member

skyluc commented Aug 14, 2014

I don't quite get why SaveActionExtension and the saveaction.*Creators have to be that complex. My guess is to have a unique perform() method in SaveAction. But it doesn't make the code simple, it won't be possible to create an extension point with this API, and the comment Do not implement this value by any means! It will be automatically implemented by the IDE is basically a lie.

For me, it would make more sense to have an API like :

trait SaveAction extends ScalaIdeExtension {
}

trait SaveActionWithDocument extends SaveAction {
  def perform(document: Document)
}

trait SaveActionWithCompiler extends SaveAction {
  def perform(document: Document, c: ScalaPresentationCompiler, t: ScalaPresentationCompiler#Tree, sf: SourceFile,)
}

Then, it is only a matter of pattern matching to know which perform method to call.

ext
}.getOrElse(Seq())
}
Await.ready(f, Duration.Inf).value.get match {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using the TimeoutFuture is redundant of using the Await.ready with the right timeout.
You can have a simple Future, and the result of ready would be None in case of timeout, Some(Success(..)) on success, Some(Failure(e)) on execption e during the withSafeRunner.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I already use Await.ready internally in TimeoutFuture, but I need to wait on two different futures and take the first one that completes, therefore I don't see how to simplify it.

@skyluc
Copy link
Member

skyluc commented Aug 14, 2014

Why do we need a wrapper for IDocument and a new changes hierarchy? We should be able to use IDocument, DocumentChange and TextEdit. Normally the even the refactoring changes become TextEdit at some point.

@kiritsuku
Copy link
Member Author

An explanation about the design of the API:

The current design originated from my extensions project, it is an API meant to be used by end-users that don't have a lot of knowledge about Eclipse (and are not meant to to have). Beside from that my goal is to provide an API that does not only work for save actions, but also for other features.

We only have save actions at the moment and the extensions that can be created by users do not yet work. That is why the design looks a bit heavy at the moment, but I believe that it is better to look ahead and implement the API accordingly - otherwise we would have a lot of changes to the API afterwards once my other ideas are implemented.

My thoughts about specific points:

  • All dependencies are injected by the IDE, they should not be seen by the implementor of the ScalaIdeExtension subclasses, therefore the perform() methods don't take any parameters. The *Creator classes do inject the dependencies but they exist only because there is no mechanism yet that can load the extensions dynamically at runtime (therefore there is no real injection happening at the moment and the comments about Do not implement this... are a leftover of that idea).
  • The Document trait should abstract over the internals. Eclipse, JDT, scala-refactoring, scalariform have all their own text change objects and abstractions for a document, I don't want that users have to see all of them. Because Document is immutable, there are less ways to destroy the content of the document. I'm not sure yet if it makes sense to keep the Change hierarchy, I have to spend more time writing save actions to find out if they are useful.
  • I'm not sure if it makes sense to support Eclipse extension points in the long term. They are very heavyweight and I hope in being able to design something simpler. I already got a prototype of simpler extensions up and running (and it can already replace the *Creation types), but there are too many unsolved problems yet.

@ghprb-bot
Copy link

Test PASSed.
Refer to this link for build results: https://jenkins.scala-ide.org:8496/jenkins/job/ghprb-scala-ide-validator/977/

@AfterClass
final def deleteProject(): Unit = {
EclipseUtils.workspaceRunnableIn(ScalaPlugin.plugin.workspaceRoot.getWorkspace()) { _ =>
project.underlying.delete(/* force */ true, new NullProgressMonitor)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is already implemented in SDTTestUtils.deleteProject().

@skyluc
Copy link
Member

skyluc commented Aug 22, 2014

The feature looks really good. I still have a few manual test I want to do.

My main problem is with the side stuff.

Document and Change - I have a hard time to see the advantage of having the new hierarchy in comparison with the one provided by Eclipse (IDocument and TextEdit). The other abstractions are coming from the outside libraries, and at some point they are mapped to the Eclipse ones.
You speak about immutability, but the implementation is backed by an IDocument, which is mutable.
The only advantage is that it is simpler and more Scala-like than the Eclipse abstraction. But it also means that dev who know Eclipse would have to learn something different.

a new extension system - I see a lot of difficult problems to solve to be able to create something different next to the Eclipse extension system. I think it has to be down at the osgi services level to be able to pick up configuration from other plugins and to link everything together, and correctly with the classloaders and other fun things.
I wouldn't mind to see your solution, but I don't think it belongs inside Scala IDE. The main goal of the project is to support Scala and its ecosystem inside Eclipse. Maintaining a new extension system is not part of that.

@kiritsuku
Copy link
Member Author

About Document and Change: I'm also not sure if they really make the world better, especially because they are not heavily used in this branch. Please have a look at my auto edits branch, which makes more use of the API. At this branch I have already more Change subclasses. It feels to be more useful there because the additional abstraction makes the auto edit implementation more descriptive.

I also think the new design is more extensible in the long term, for example it allows us to have support for linked models, which is not possible with IAutoEditStrategy (one of the main reasons why I already have the plan for a long time to move away from this class).

@skyluc
Copy link
Member

skyluc commented Aug 22, 2014

I have a test case where the caret position is not where I would have expected it after the save actions are run. I have all of them enabled.

Before

package test
class ScalaClass {
  /**
   */  ^$
  $
}

After

package test
class ScalaClass {
  /**
   ^*/

}

I would expected the caret to be after the closing comment tag */, not before.

@skyluc
Copy link
Member

skyluc commented Aug 22, 2014

The 'undo' is not right. It should work as one big change to revert to the content just before the save actions are run. Right now, I have to hit undo for each change made to the file.

@ghprb-bot
Copy link

Test FAILed.
Refer to this link for build results: https://jenkins.scala-ide.org:8496/jenkins/job/ghprb-scala-ide-validator/988/

@kiritsuku kiritsuku mentioned this pull request Aug 25, 2014
5 tasks
@dragos
Copy link
Member

dragos commented Sep 1, 2014

I'd like to give it a go locally, could you please rebase? It doesn't merge cleanly anymore...

@dragos
Copy link
Member

dragos commented Sep 2, 2014

This is a great feature, and I love the attention to detail and new features you managed to implement so quickly!

However, in my opinion the main appeal of this feature is the ability for users to write their own save actions "on the fly", in an editor or edit box, and have them running without restarting the IDE. I see that as the main goal of this project. The value comes from user-configurability. Even if there are still certain limitations (maybe tree-based actions won't be possible), this is the holy grail. What do you think?

@dragos
Copy link
Member

dragos commented Oct 9, 2014

Awesome!!

Conflicts:
	org.scala-ide.sdt.core.tests/src/org/scalaide/TestsSuite.scala
- Refactoring based save actions are disabled because they are too
  unstable
- `Document.Range` is removed because it is unnecessary
- Bounds checking is removed from `Document`

With the introduction of save actions some open tickets that document
problems in combination with the JDT save action feature can be marked
as fixed because JDT is no longer involved in the Scala editor with its
save actions.

Re #1000499
Fixes #1000900
Fixes #1000887
Fixes #1001138
@kiritsuku
Copy link
Member Author

I solved most points (see last commit), but not yet the Undo behavior. It turned out to be more difficult to do this in general. Also, I found a NPE that sometimes occurs but I couldn't find out so far why it happens. It somehow occurs when switching between Scala editors. Open points:

  • Fix undo behavior
  • Fix NPE
  • Adapt the new quick fixes for this PR

@dragos
Copy link
Member

dragos commented Oct 9, 2014

@sschaef Am I correct that you wish this PR to be merged before quick fixes?

@kiritsuku
Copy link
Member Author

No afterwards. Quick fixes don't work anymore once this PR is merged and fixing the quick fix PR first means that we can verify that it is really possible to fix them...

@ghprb-bot
Copy link

Test FAILed.
Refer to this link for build results: https://jenkins.scala-ide.org:8496/jenkins/job/ghprb-scala-ide-validator/1219/

@dragos
Copy link
Member

dragos commented Oct 9, 2014

Actually, if the number of undo actions that one needs to completely revert save actions is equal to the number of enabled save actions, I don't think it's a huge deal. It would be 1, 2 or maximum three, but I don't see why would anyone enable all three of them (Formatting already removes space at the end of the line) :)

@dragos
Copy link
Member

dragos commented Oct 9, 2014

(ignore the test failure, that's what I was trying to fix in #828)

@kiritsuku
Copy link
Member Author

Actually it is worse than that. I found out that the merged text changes is not a real merged text change, it is just a wrapper for multiple changes. Therefore we still have as many undos as text changes.

@dragos
Copy link
Member

dragos commented Oct 9, 2014

I just peeked at how the Scala formatter does it:

val multiEdit = new MultiTextEdit
multiEdit.addChildren(edits.map(asEclipseTextEdit).toArray)
val change = new CompilationUnitChange("Formatting", cu)
change.setEdit(multiEdit)
new CodeFormatFix(change)

CompilationUnitChange implements some Undo logic. It's part of the Eclipse public API, so we could use it, if it fits the purpose.

@kiritsuku
Copy link
Member Author

Ok, I'll have a look.

Each changed region creates its own undo entry in the undo queue. This
means that one has to undo each text change separately after save
actions are invoked instead of only a single undo.

Fortunately it is possible to tell the undo queue that multiple undos
belong together.
Conflicts:
	org.scala-ide.sdt.core/plugin.xml
	org.scala-ide.sdt.core/src/org/scalaide/util/eclipse/EclipseUtils.scala
It is no longer possible to retrieve compilation units from JDT classes,
we have to use the definitions in `IScalaPlugin` and `ScalaPlugin`.
@kiritsuku
Copy link
Member Author

Good news: It's done! The Undo thing and quick assists work fine. There is just the NPE left but I have still no clue on how to reproduce it. How awesome that this is just RC1...

I'm not sure why this exception occurs but I guess that whenever an
editor is closed, its compilation unit is removed but the job for the update occurrences feature may still be running. We just need to check if null is returned and do nothing in this case.
@kiritsuku
Copy link
Member Author

NPE fixed. The only important thing that is missing now is documentation.

@dragos
Copy link
Member

dragos commented Oct 9, 2014

What do you think about removing the refactoring-based tests until we iron out the details?

Results :

Failed tests:   add_override_modifier_to_constructor_param(org.scalaide.extensions.saveactions.AddMissingOverrideTest): expected:<... }

@kiritsuku
Copy link
Member Author

Yeah, seems to be better. I make a party the day when I find out why exactly all these compiler tests fail.

@misto
Copy link
Member

misto commented Oct 9, 2014

I'll buy the drinks for that party ;-) Great work Simon!

@ghprb-bot
Copy link

Test PASSed.

See Console Output in the link below for an update site containing this PR binary artefacts.

Refer to this link for build results: https://jenkins.scala-ide.org:8496/jenkins/job/ghprb-scala-ide-validator/1227/

@dragos
Copy link
Member

dragos commented Oct 9, 2014

It looks good! Great work, Simon!

It's still a bit slow, but I don't know if we can fix that... I timed different parts of the implementation and almost all the time is spent in multiEdit.apply(document). On a large file with many empty lines (>1000 lines) you can see very nicely that edits are applied in sequence (and the editor is refreshed after each one). I think that's what slows it down, and it would be great to fix it. There must be another way to apply edits, since Java save actions (surprisingly, they still work on Scala files too) don't have such behavior.

I don't think that's a blocker for RC1, though! I'd just like to keep this in mind, and try to fix it for the final release.

Again, fantastic work and thanks for the dedication to pull this through!

dragos added a commit that referenced this pull request Oct 9, 2014
@dragos dragos merged commit 4cce06b into scala-ide:master Oct 9, 2014
@kiritsuku kiritsuku deleted the save-actions branch October 9, 2014 18:52
@skyluc
Copy link
Member

skyluc commented Oct 9, 2014

One idea. Documents should have a 'working copy' system. It might speed thing up to do it on a working copy, and to apply it to the real document.
It doesn't make sense that the changes can be seen being applied in the UI.

@kiritsuku
Copy link
Member Author

Yes, updating the UI shouldn't happen when it is not necessary. There is still some time before the final release, I'll find a way to make it more performant.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants