WIP: Pretty error printer for predicates. #39

XAMPPRocky · 2018-05-28T15:17:15Z

Currently missing implements for a lot of the predicates currently only AndPredicate, OrdPredicate, and EqPredicate have correct impls.

Table Output

╔═══════════════════════╦══════════╗
║ PREDICATE             ║ ROW      ║
╠═══════════════════════╬══════════╣
║ !(5 != 5) && 5 >= 5   ║ PASSED   ║
╠═══════════════════════╬══════════╣
║ !(5 != 5)             ║ PASSED   ║
╠═══════════════════════╬══════════╣
║ 5 != 5                ║ FAILED   ║
╠═══════════════════════╬══════════╣
║ 5 >= 5                ║ PASSED   ║
╚═══════════════════════╩══════════╝

Tree Output

!(5 != 5) && 5 >= 5 PASSED
├── !(5 != 5) PASSED
|   └── 5 != 5 FAILED
└── 5 >= 5 PASSED

nastevens · 2018-05-28T17:11:21Z

Thanks for the PR @Aaronepower. Unfortunately, I'm worried this is a non-starter as it's currently written: I am quite strongly against requiring that all Items implement Debug. The reason is that Item should be allowed to be literally anything, and not every type will have a Debug implementation. We need to be as generic as possible in what types work with Item to allow other libraries to work with the predicates library in as generic a fashion as possible.

I'd be much more receptive to this idea if there were a way to make it opt-in, i.e. not require Debug but use it if it's available. I'm not immediately seeing a way to do that unfortunately, but you might have some ideas.

epage · 2018-05-29T07:24:22Z

Thanks for the PR @Aaronepower. Unfortunately, I'm worried this is a non-starter as it's currently written: I am quite strongly against requiring that all Items implement Debug. The reason is that Item should be allowed to be literally anything, and not every type will have a Debug implementation. We need to be as generic as possible in what types work with Item to allow other libraries to work with the predicates library in as generic a fashion as possible.

@nastevens, Aaron and I had talked about this before he posted this. It is very rare that Debug isn't supported and if it isn't, it should probably be added.

Do you have an example of a case that'd be a problem?

If this doesn't work out, we could turn this into an additional trait but that will get kind of messy, like expanding AndPredicate and so on. I had considered this originally but saw no value, so I decided against it.

epage · 2018-05-29T07:29:04Z

src/boolean.rs

@@ -21,7 +21,7 @@ pub struct AndPredicate<M1, M2, Item>
 where
    M1: Predicate<Item>,
    M2: Predicate<Item>,
-    Item: ?Sized,
+    Item: ?Sized + fmt::Debug,


Thanks for making this a dedicated commit

If we keep this, please note in the commit message that you are breaking the API.

Personally, I recommend following conventional changelog, it makes it easier for me to notice everything when deciding on the new version / writing the changelog.

epage · 2018-05-29T07:29:48Z

src/main.rs

@@ -0,0 +1,7 @@
+extern crate predicates;


Why is this in the current commit?

epage · 2018-05-29T07:30:08Z

src/core.rs

    /// Execute this `Predicate` against `variable`, returning the resulting
    /// boolean.
    fn eval(&self, variable: &Item) -> bool;
+
+    /// TODO
+    fn eval_to_string(&self, variable: &Item) -> String {


Why is this in the current commit?

epage · 2018-05-29T07:32:22Z

src/core.rs

+
+    /// TODO
+    #[cfg(feature = "term-table")]
+    fn tree_eval(&self, variable: &Item) -> String {


We need a way to programmatically know the result of the evaluation, so a caller like assert_cmd can decide whether to panic or not.

XAMPPRocky · 2018-05-29T07:59:40Z

@nastevens Thanks for your feedback. There are types that don't implement Debug they also don't tend to implement PartialEq or PartialOrd which as far as I can tell would prevent them from using this library. Is there a use case where a type would be able to be compared and would not be able to have a debug representation?

epage · 2018-05-29T08:25:04Z

Background: This is an implementation for #7 (btw if you note that in commit message as Fixed #7, github will auto-close the issue) so that crates like assert_cmd and assert_fs can describe to the user why the operation failed so the user can start looking into the failure without the need for a debugger.

To get an idea of what I'm talking about, here is an artificial failure in, what to me, is the most ideal test runner I've ever used

___________________________________________________________ test_session_container ___________________________________________________________
                                                                                                                                              
    def test_session_container():                                                                                                             
>       helper(pathlib.Path(__file__), "bye")                                                                                                 
                                                                                                                                              
tests/test_convert.py:24:                                                                                                                     
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
                                                                                                                                              
one = PosixPath('/home/epage/git/nixnet-python/tests/test_convert.py'), two = 'bye'                                                           
                                                                                                                                              
    def helper(one, two):                                                                                                                     
        expected_one = 5                                                                                                                      
        expected_two = "hello"                                                                                                                
>       assert expected_one == one.parent.parent.with_suffix(".foo") and expected_two == two                                                  
E       AssertionError: assert (5 == PosixPath('/home/epage/git/nixnet-python.foo'))                                                          
E        +  where PosixPath('/home/epage/git/nixnet-python.foo') = <bound method PurePath.with_suffix of PosixPath('/home/epage/git/nixnet-python')>('.foo')                                                                                                                                
E        +    where <bound method PurePath.with_suffix of PosixPath('/home/epage/git/nixnet-python')> = PosixPath('/home/epage/git/nixnet-python').with_suffix                                                                                                                              
E        +      where PosixPath('/home/epage/git/nixnet-python') = PosixPath('/home/epage/git/nixnet-python/tests').parent                    
E        +        where PosixPath('/home/epage/git/nixnet-python/tests') = PosixPath('/home/epage/git/nixnet-python/tests/test_convert.py').parent                                                                                                                                          
                                                                                                                                              
tests/test_convert.py:20: AssertionError

This shows that the test fails and shows what the subexpessions evaluate to in a tree structure for easy parsing by humans.

My original plan for this trait method was to find counter examples (passing in a bool), returning an iterator of the failures. Each failure would generate a similar evaluation tree, possibly using treeline crate. The value of it returning an iterator is for cases like dir-diff where someone might just care about the first failure or they might want to know all the failures because the first might not be the most informative. As for implementation, my thought is it would return borrows

My thought is that the Predicate Display would not show struct members but variables.

The downsides to this

A bit complex
A lot allocations because each iterator is probably a Vec and as we unwind the stack, we'll need to destroy the Vec we've received and return a new Vec in its place

Now let's see how well I remember my conversation with @Aaronepower from yesterday. He wanted to start with a simpler implementation. For example, instead of visually representing it as a tree, what if we just list each sub expression in series. As for Predicate struct members, Aaron was concerned about repeated names and it being confusing (if a name appears twice, is it the same thing?). This led him to wanting to show the actual content. The plan is to ellipsis it if it overflows ad to add a row expanding the ellipsised content.

For now, we have a vertical slice to iterate on the design to minimize churn.

epage · 2018-05-29T08:29:29Z

src/core.rs

@@ -8,15 +8,60 @@

 use std::fmt;

+struct TreeWriter<'a, Item: ?Sized + fmt::Debug + 'a> {


Dead code?

Speaking of, should we have a ExpressionWriter trait that flatten accepts as a reference to decouple the allocation policy from the Predicate trait?

epage · 2018-05-29T08:31:56Z

src/core.rs

    /// Execute this `Predicate` against `variable`, returning the resulting
    /// boolean.
    fn eval(&self, variable: &Item) -> bool;
+
+    /// TODO
+    fn flatten<'a, 'b>(&'a self, _vec: &'b mut Vec<&'a Predicate<Item>>) {


So, what name should we use for this?

epage · 2018-05-29T08:32:45Z

src/core.rs

+    }
+
+    /// TODO
+    fn stringify(&self, _variable: &Item) -> String {


So, what name should we use for this?

The difference between this and Display is that this includes and evaluation of var / actual.

epage · 2018-05-29T08:33:07Z

src/ord.rs

 }

 impl<T> fmt::Display for EqPredicate<T>
 where
    T: fmt::Debug,
 {
    fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
-        write!(f, "var {} {:?}", self.op, self.constant)
+        write!(f, "{} {:?}", self.op, self.constant)


Why is var removed?

XAMPPRocky · 2018-05-29T10:35:20Z

I've added a commit that switches it from a table output to a tree output so that we can compare and contrast bwetween the two.

nastevens · 2018-05-29T11:41:16Z

I admit I am having a really hard time thinking of an example in my own usage where requiring Debug would break anything. At the same time, as API designers, we need to be thinking about more than just our own usage and be extremely wary of putting extra constraints on what the API can handle. "Be liberal in what your API accepts and conservative in what it returns".

That said, I'm coming around to the idea that this change is a minor step backwards (API is slightly more restrictive in what it accepts) for a major step forward (a human can actually get visibility into their Predicate objects). All-in-all, I still have some reservations but without being able to either give specific breaking examples or come up with a better suggestion, I'm giving this approach a 👍

epage · 2018-06-16T17:04:02Z

Sorry I'm just now getting back to this. I've started to record in #7 the different lessons we requirements and issues we learned through this. I'll hopefully get back to this again soon with either more feedback or yet another idea for us to try.

Finnaly, Fixes assert-rs#7. Inspired by the work in assert-rs#39. Created softprops/treeline#3 for trying to find ways to make this more efficient.

epage · 2018-07-21T22:18:53Z

Again, thanks @Aaronepower for your work on this. This helped a lot in uncovering requirements (recorded in #7) and helped pave the way for #60. Unless there is concern otherwise, I'm considering #60 to supersede this PR.

Finnaly, Fixes assert-rs#7. Inspired by the work in assert-rs#39. Created softprops/treeline#3 for trying to find ways to make this more efficient.

Aaron Power added 2 commits May 28, 2018 12:56

Added fmt::Debug requirement to Item

9273572

(feat): Added a pretty error printer

a5b27e8

epage requested changes May 29, 2018

View reviewed changes

feat(tree_eval): Additional implementations & signature change

ccc0414

epage requested changes May 29, 2018

View reviewed changes

Changed output to make use of treeline over term-table

c16f2fc

epage mentioned this pull request Jun 16, 2018

An eval that tells me why the predicate failed #7

Closed

epage added a commit to epage/predicates-rs that referenced this pull request Jul 21, 2018

feat: Tree view of Predicate result (Case)

6cc1a13

Finnaly, Fixes assert-rs#7. Inspired by the work in assert-rs#39. Created softprops/treeline#3 for trying to find ways to make this more efficient.

epage mentioned this pull request Jul 21, 2018

A way to display *why* a predicate failed #60

Merged

epage closed this Jul 21, 2018

epage added a commit to epage/predicates-rs that referenced this pull request Jul 21, 2018

feat: Tree view of Predicate result (Case)

34e7f37

Finnaly, Fixes assert-rs#7. Inspired by the work in assert-rs#39. Created softprops/treeline#3 for trying to find ways to make this more efficient.

epage added a commit to epage/predicates-rs that referenced this pull request Jul 28, 2018

feat: Tree view of Predicate result (Case)

a9647cd

Finnaly, Fixes assert-rs#7. Inspired by the work in assert-rs#39. Created softprops/treeline#3 for trying to find ways to make this more efficient.

epage added a commit to epage/predicates-rs that referenced this pull request Jul 28, 2018

feat: Tree view of Predicate result (Case)

188a847

Finnaly, Fixes assert-rs#7. Inspired by the work in assert-rs#39. Created softprops/treeline#3 for trying to find ways to make this more efficient.

epage added a commit to epage/predicates-rs that referenced this pull request Jul 28, 2018

feat: Tree view of Predicate result (Case)

67085fc

Finnaly, Fixes assert-rs#7. Inspired by the work in assert-rs#39. Created softprops/treeline#3 for trying to find ways to make this more efficient.

		@@ -8,15 +8,60 @@

		use std::fmt;

		struct TreeWriter<'a, Item: ?Sized + fmt::Debug + 'a> {

WIP: Pretty error printer for predicates. #39

WIP: Pretty error printer for predicates. #39

Uh oh!

Conversation

XAMPPRocky commented May 28, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Table Output

Tree Output

Uh oh!

nastevens commented May 28, 2018

Uh oh!

epage commented May 29, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

XAMPPRocky commented May 29, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

epage commented May 29, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

XAMPPRocky commented May 29, 2018

Uh oh!

nastevens commented May 29, 2018

Uh oh!

epage commented Jun 16, 2018

Uh oh!

epage commented Jul 21, 2018

Uh oh!

Uh oh!

XAMPPRocky commented May 28, 2018 •

edited

Loading

epage commented May 29, 2018 •

edited

Loading

XAMPPRocky commented May 29, 2018 •

edited

Loading

epage commented May 29, 2018 •

edited

Loading