Skip to content
This repository was archived by the owner on Oct 30, 2023. It is now read-only.

Conversation

@yukselakinci
Copy link
Contributor

Second PR for serialization. There is one more.

*
* @param <T> Object type
*/
public class KryoSimpleWrapper<T> implements Writable {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the difference between this and KryoWritableWrapper, when should we use one and when the other?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have added more documentation now. The usage of KryoSimpleWrapper is similar to KryoWritabeWrapper. KryoSimpleWrapper can be used if the object being serialized is not recursive or nested, because it uses a version of kryo object that doesn't track references, hence significantly faster.

* This serialization does not support recursive/nested structures
* to provide faster serialization.
*/
public abstract class KryoSimpleWritable implements KryoIgnoreWritable {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be great to switch to Java 8, to be able to keep this an interface :-)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree :-)

// non-kryo serialization.
/** Reusable Input object */
private final InputChunked input = new InputChunked(4096);
private InputChunked input;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we separate two Kryo implementations in two classes since they have a lot of differences?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, we can refactor the existing code to have a better code readability. I have added a TODO for that.

* no recursive/nested objects should be serialized.
* @return Hadoop kryo which doesn't track objects.
*/
public static HadoopKryo getNontrackingKryo() {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How will someone using kryo know which implementation to use? Can we add some guidance around that?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added more documentation.

Copy link

@majakabiljo majakabiljo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed offline, @yukselakinci will in a separate diff refactor HadoopKryo.

@asfgit asfgit closed this in 60752aa Apr 12, 2018
@yukselakinci yukselakinci deleted the phase22 branch April 17, 2018 16:45
dlogothetis pushed a commit to dlogothetis/giraph that referenced this pull request Apr 26, 2018
dlogothetis pushed a commit to dlogothetis/giraph that referenced this pull request Apr 26, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants