[A3] The bowels of a Message

Moderator: Praktikum: Internet

Geek-O
Windoof-User
Windoof-User
Beiträge: 34
Registriert: 16. Dez 2004 22:42

[A3] The bowels of a Message

Beitrag von Geek-O » 20. Jan 2008 17:47

Okay,
it seems I am totally at my wit's end here:

(1) How to completely serialize a Message object?
I am trying to turn a message into a byte array which I can transfer and then rebuild an identical message at the other side.

What I currently understood is this:
A message can basically contain three types of chunks: active, passive and bin. We don't have to care about active ones because a message will already have been passivized by the time it reaches our handler. So we can use BinSerializer.serializeAll(Message) to move all passive and bin chunks to all:bin; this even retains the old contents of all:bin, so BinDeserializer.deserialize(Message) should bring the message back to its original state from just the contents of the all:bin Blob.
Unfortunately, the rs and ts chunks are not, as the JavaDoc for Message states, of type passive, but of type param, so serializeAll() does not handle them. I could try to save them separately, but what if someone invents another chunk of a type which is then also not serialized? Or are there any chunks I can safely drop anyway?


(2) How to create a new message that will be sent to the same destination as a given message? Which chunks do I need to copy?

Currently I clone the original message, then remove all chunks I know to be irrelevant for adressing and routing. But again, I can only remove chunks whose name I know, so what if someone invents a new one...?


(3) For the fragmentation milestone, which MTU value do I use? Should I just allow a compile time configuration, do PMTU discovery, or something in between? And how would I do that, does MundoCode help me in any way?


Generally, my feeling is that the main problem of this third assignment is not implementing the neccessary functionality, but coping with a given library which not only has inadequate (and -obviously- partially wrong) documentation, but also tends to give pretty useless error messages (What good is a NullPointerException without any additional information, especially if I can't even find the source line where it was thrown because the available code is not in .java but in some .oj form that needs preprocessing I don't know how to apply? (Just calling mcc on the file didn't help.) -- Okay, so it's OpenJava. Now what would that be?)

If I sound frustrated, that's because I am, especially after going to great lengths to turn assignment 2 in on time, only to find the deadline being extended at the last minute.
This might be an interesting challenge, but not with so little time and so close to the end of the semester, where there also are exams to be prepared.

G., missing the second weekend in a row

erwin.ait
Erstie
Erstie
Beiträge: 20
Registriert: 23. Jul 2007 16:34

Re: [A3] The bowels of a Message

Beitrag von erwin.ait » 23. Jan 2008 12:48

(1)
Use a stack like shown on Slide 36.
We assume that SampleHandler1 is your handler.
The BinCollectHandler above your handler will collect everything that should be transported to the remote peer into the chunk "all:bin".
For that reason, you only need to care about "all:bin".
To read/write your own header, use AbstractHandler.putHeader and AbstractHandler.getHeader.
The BinCollectHandler below your handler (Slide 36) will put all chunks of type "bin" into "all:bin". The transport service will then simply send "all:bin" to the peer.
The "param" chunks are used to pass parameters to protocol handlers locally and they are not meant to be sent to remote peers.

(2)
clone the Message and replace "all:bin".
At this point you do not need to care about other chunks, because BinCollectHandler will take care of them.

(3)
We'll just use a fixed MTU.
The MTU should be configurable in the configuration file.
See Slide 41 and the source code of DropHandler how to do that.

Some general comments:

I think a lot of that has been explained in the lecture and in the example programs.
I must admit that I've made a number of simplifications and I did not explain everything, but please
- Read the slides
- Look at the example programs ex1..ex5 and praktikum
- Use org.mundo.net.AbstractHandler. It has helper functions that take care of the subtleties of serialization, etc.

For debugging, use prep.zip. This archive contains the Java source code as generated by the precompiler from the OJ-source.
When you discover wrong documentation: please tell me, because we want to fix that.

I understand that this assignment is not trivial and I'm aware of the usual end-of-semester stress.
That's why we've started with the third assignment already in December.
If I sound frustrated, that's because there were more than four weeks time and all the "good questions" come one week before the deadline.

Geek-O
Windoof-User
Windoof-User
Beiträge: 34
Registriert: 16. Dez 2004 22:42

Re: [A3] The bowels of a Message

Beitrag von Geek-O » 25. Jan 2008 15:29

Hi, and thanks for your reply.
erwin.ait hat geschrieben: The BinCollectHandler above your handler will collect everything that should be transported to the remote peer into the chunk "all:bin".
For that reason, you only need to care about "all:bin".
[...]
The BinCollectHandler below your handler (Slide 36) will put all chunks of type "bin" into "all:bin". The transport service will then simply send "all:bin" to the peer.
That was it! I have no idea why, suddenly after reading your explanation my handler worked perfectly after just removing the BinSerializer and BinDeserializer lines and otherwise not changing anything, when before I got all kinds of odd exceptions without them. Seems to be one of those cases where you "fix" a problem by adding another mistake, then make more changes until the only problem that's left is the original "fix".
The "param" chunks are used to pass parameters to protocol handlers locally and they are not meant to be sent to remote peers.
Still, I found that without them, the message would go nowhere.
(2)
clone the Message and replace "all:bin".
At this point you do not need to care about other chunks, because BinCollectHandler will take care of them.
I'd rather not do that. The messages that are passed down to me look like this:

Code: Alles auswählen

payload:bin:. \0eS`!Y\1c\1b]My<75{gH\02i,\07]5I\17eBXiqx? k\02)\15(\03CH\1c/\02[if\0b.\0b8C+;\0ba>LATlZ+[...]
main:passive:{org.mundo.rt.TypedMap: seq=3}
all:bin:\11\00\00\00message/mc-binser\02\00\00\00\fa\03\00\00\07\00\00\00payload\03\00\00\00bin. \0eS`!Y[...]
rs:param:{destId=0AFF344D88FB96D5248D90E3B501DB3B, destType=node}
ers:param:{channel=nettest}
main:active:{seq=3}
ers:passive:{request=message, seq=2, contentType=(null), channel=nettest}
ts:param:{route=0AFF..DB3B tcp://127.0.0.1:4245}
address:passive:{mimeType=message/mc-bincoll}
If I just replace all:bin, I'd still be passing lots of unnecessary data (here esp. payload:bin) on. (This was also the reason I was confused about Serialization and the role of the BinCollectHandler: I had assumed that if BCH's job was to pack (move) all the content into all:bin, then why would all those non-routing chunks still be there?)

I have meanwhile figured out a way by creating a new message, then using copyStackFrom() and copying the ts:param and rs:param chunks, if they exist.This seems to be working - or is there a problem with this approach?
(3)
We'll just use a fixed MTU.
The MTU should be configurable in the configuration file.
See Slide 41 and the source code of DropHandler how to do that.
Okay, that was easy enough. I did have a problem figuring out how much room to leave for headers that might be added on the lower layers (especially since layering is configurable), but if I understood correctly in the Q&A session, we are only supposed to look at all:bin anyway.
I think a lot of that has been explained in the lecture and in the example programs.
Well, styles and preferences differ and there's no point trying to discuss that, but to give some (hopefully) constructive feedback, these are some points that I would have liked to be mentioned (or for those that were mentioned, been brought forward more):
  • we only need to look at all:bin thanks to the BinCollectHandler
  • what routing and other technical information is associated with a message
  • the tutorial you talked about was not the tutorial.pdf file in the docs directory (which is instead written from an application programmer's perspective)
  • documentation may be partly lacking, so it is absolutely necessary that we read the MundoCore sources which can be found in a readable form in prep.zip (this may seem obvious, but OTOH I learned that when using a given library, I should not rely on its internal workings but on the functions it officially offers)
For debugging, use prep.zip. This archive contains the Java source code as generated by the precompiler from the OJ-source.
Ah, I have no idea why I overlooked that archive. This makes things so much easier, having readable code with line numbers matching the ones from the exception stack traces...
When you discover wrong documentation: please tell me, because we want to fix that
It may not actually be wrong (and if only due to the word "might", more probably because it depicts a different layer), but I was thoroughly confused by the table showing the chunks of a message in the Message JavaDoc: Instead of "rs.passive" and "ipts:passive" chunks, I found "rs:param" and "ts:param", and the type of the "all" chunk was not "binser", but just "bin".

Another reason for my confusion about serialization.
I understand that this assignment is not trivial
To be very honest, now that I have gotten over the starting problems I feel that this assignment is easily at least one level of magnitude easier (both difficulty-wise and in terms of lines of code required) than A2. Not that I'm complaining, since A2 turned out be to quite a fat child...
and I'm aware of the usual end-of-semester stress.
That's why we've started with the third assignment already in December.
If I sound frustrated, that's because there were more than four weeks time and all the "good questions" come one week before the deadline.
I would have been more than happy to start this assignment around Christmas (as we had originally planned), but as I said A2 brought a much higher workload than I had expected - and I don't seem to be the only one who felt like that, given that the deadline had to be extended at the last minute.
(BTW, you don't . :) )

Thanks anyway,
G.

Antworten

Zurück zu „Praktikum: Internet“