Skip to content

Issue #177 malformed urls inside json response #178

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from
Closed

Issue #177 malformed urls inside json response #178

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
8a710e4
Flag for disable response HTML encoding while json marshaling lambda …
andreas-westberg-skf Mar 10, 2019
1f4ec05
change marshalWithDisabledEscapeHTML to private
andreas-westberg-skf Mar 10, 2019
49fa0d3
Merge branch 'master' into master
bmoffatt May 22, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions lambda/handler.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,12 @@
package lambda

import (
"bytes"
"context"
"encoding/json"
"fmt"
"reflect"
"sync"

"github.com/aws/aws-lambda-go/lambda/handlertrace"
)
Expand All @@ -18,6 +20,9 @@ type Handler interface {
// lambdaHandler is the generic function type
type lambdaHandler func(context.Context, []byte) (interface{}, error)

var responseDisableEscapeHTML = false
var muresponseDisableEscapeHTML sync.Mutex

// Invoke calls the handler, and serializes the response.
// If the underlying handler returned an error, or an error occurs during serialization, error is returned.
func (handler lambdaHandler) Invoke(ctx context.Context, payload []byte) ([]byte, error) {
Expand All @@ -26,6 +31,10 @@ func (handler lambdaHandler) Invoke(ctx context.Context, payload []byte) ([]byte
return nil, err
}

if responseDisableEscapeHTML {
return marshalWithDisabledEscapeHTML(response)
}

responseBytes, err := json.Marshal(response)
if err != nil {
return nil, err
Expand All @@ -34,6 +43,19 @@ func (handler lambdaHandler) Invoke(ctx context.Context, payload []byte) ([]byte
return responseBytes, nil
}

//marshalWithDisabledEscapeHTML uses json.NewEncoder instead of json.Marshal to disable escaping HTML since it may corrupt response.
func marshalWithDisabledEscapeHTML(response interface{}) ([]byte, error) {
var buf bytes.Buffer
enc := json.NewEncoder(&buf)
enc.SetEscapeHTML(false)
err := enc.Encode(response)
if err != nil {
return nil, err
}

return buf.Bytes(), nil
}

func errorHandler(e error) lambdaHandler {
return func(ctx context.Context, event []byte) (interface{}, error) {
return nil, e
Expand Down Expand Up @@ -72,6 +94,15 @@ func validateReturns(handler reflect.Type) error {
return nil
}

// SetResponseNonEscapeHTML will NOT coerced JSON strings to valid UTF-8,
// The angle brackets "<" and ">" will NOT be escaped to "\u003c" and "\u003e"
// Ampersand "&" is also NOT be escaped to "\u0026".
func SetResponseNonEscapeHTML(flag bool) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of a package global setter, I'd prefer optional values to somehow be configured on the type returned by NewHandler. I think this could be done in a backwards compatible way in one of these two approaches:

  1. adding a HandlerOptions type to pass as varargs to NewHandler. eg:
  • NewHandler(handler interface{}, opts... HandlerOptions) lambda.Handler.
  1. update NewHandler to return a new struct *JSONHandler still implementing lambda.Handler, and add this add this configuration as a method for that struct. eg:
  • func NewHandler(handler interface{}) *JSONHandler
  • func (handler *JSONHandler) SetEscapeHTML(on bool)

I think I like option 2 better, since it would setup a pattern for mirroring the other json encode/decode options like https://golang.org/pkg/encoding/json/#Encoder.SetEscapeHTML

In-use, this'd look like

func main() {
        jsonHandler := NewHandler(func () (string, error) {
                return "<html><body>html in json string!</body></html>", nil
        });
        jsonHandler.SetEscapeHTML(false);
        lambda.StartHandler(jsonHandler)
}

muresponseDisableEscapeHTML.Lock()
responseDisableEscapeHTML = flag
muresponseDisableEscapeHTML.Unlock()
}

// NewHandler creates a base lambda handler from the given handler function. The
// returned Handler performs JSON deserialization and deserialization, and
// delegates to the input handler function. The handler function parameter must
Expand Down
54 changes: 54 additions & 0 deletions lambda/handler_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -201,6 +201,60 @@ func TestInvokes(t *testing.T) {
}
}

func TestInvokeResponseStructs(t *testing.T) {
type testStructPerson = struct {
Name string
Age int64
Active bool
}

type testStructURL = struct {
URL string
}
testCases := []struct {
name string
disableEscapeHTML bool
expected expected
handler interface{}
}{
{
name: "basic input struct serialization ",
expected: expected{`{"Name":"Aws","Age":64,"Active":true}`, nil},
handler: func() (testStructPerson, error) {
return testStructPerson{"Aws", 64, true}, nil
},
},
{
name: "basic input struct serialization URL",
disableEscapeHTML: true,
expected: expected{`{"URL":"https://github.com/aws/aws-lambda-go?hello=world&go=lang"}` + "\n", nil},
handler: func() (testStructURL, error) {
return testStructURL{URL: `https://github.com/aws/aws-lambda-go?hello=world&go=lang`}, nil
},
},
{
name: "basic input struct serialization URL",
expected: expected{`{"URL":"https://github.com/aws/aws-lambda-go?hello=world\u0026go=lang"}`, nil},
handler: func() (testStructURL, error) {
return testStructURL{URL: `https://github.com/aws/aws-lambda-go?hello=world&go=lang`}, nil
},
},
}
for i, testCase := range testCases {
t.Run(fmt.Sprintf("testCase[%d] %s", i, testCase.name), func(t *testing.T) {
lambdaHandler := NewHandler(testCase.handler)
SetResponseNonEscapeHTML(testCase.disableEscapeHTML)
response, err := lambdaHandler.Invoke(context.TODO(), []byte(`"Lambda"`))
if testCase.expected.err != nil {
assert.Equal(t, testCase.expected.err, err)
} else {
assert.NoError(t, err)
assert.Equal(t, testCase.expected.val, string(response))
}
})
}
}

func TestInvalidJsonInput(t *testing.T) {
lambdaHandler := NewHandler(func(s string) error { return nil })
_, err := lambdaHandler.Invoke(context.TODO(), []byte(`{"invalid json`))
Expand Down