.\" Automatically generated by Pod::Man 4.10 (Pod::Simple 3.35) .\" .\" Standard preamble: .\" ======================================================================== .de Sp \" Vertical space (when we can't use .PP) .if t .sp .5v .if n .sp .. .de Vb \" Begin verbatim text .ft CW .nf .ne \\$1 .. .de Ve \" End verbatim text .ft R .fi .. .\" Set up some character translations and predefined strings. \*(-- will .\" give an unbreakable dash, \*(PI will give pi, \*(L" will give a left .\" double quote, and \*(R" will give a right double quote. \*(C+ will .\" give a nicer C++. Capital omega is used to do unbreakable dashes and .\" therefore won't be available. \*(C` and \*(C' expand to `' in nroff, .\" nothing in troff, for use with C<>. .tr \(*W- .ds C+ C\v'-.1v'\h'-1p'\s-2+\h'-1p'+\s0\v'.1v'\h'-1p' .ie n \{\ . ds -- \(*W- . ds PI pi . if (\n(.H=4u)&(1m=24u) .ds -- \(*W\h'-12u'\(*W\h'-12u'-\" diablo 10 pitch . if (\n(.H=4u)&(1m=20u) .ds -- \(*W\h'-12u'\(*W\h'-8u'-\" diablo 12 pitch . ds L" "" . ds R" "" . ds C` "" . ds C' "" 'br\} .el\{\ . ds -- \|\(em\| . ds PI \(*p . ds L" `` . ds R" '' . ds C` . ds C' 'br\} .\" .\" Escape single quotes in literal strings from groff's Unicode transform. .ie \n(.g .ds Aq \(aq .el .ds Aq ' .\" .\" If the F register is >0, we'll generate index entries on stderr for .\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index .\" entries marked with X<> in POD. Of course, you'll have to process the .\" output yourself in some meaningful fashion. .\" .\" Avoid warning from groff about undefined register 'F'. .de IX .. .nr rF 0 .if \n(.g .if rF .nr rF 1 .if (\n(rF:(\n(.g==0)) \{\ . if \nF \{\ . de IX . tm Index:\\$1\t\\n%\t"\\$2" .. . if !\nF==2 \{\ . nr % 0 . nr F 2 . \} . \} .\} .rr rF .\" ======================================================================== .\" .IX Title "Google::ProtocolBuffers 3pm" .TH Google::ProtocolBuffers 3pm "2018-11-15" "perl v5.28.0" "User Contributed Perl Documentation" .\" For nroff, turn off justification. Always turn off hyphenation; it makes .\" way too many mistakes in technical documents. .if n .ad l .nh .SH "NAME" Google::ProtocolBuffers \- simple interface to Google Protocol Buffers .SH "SYNOPSYS" .IX Header "SYNOPSYS" .Vb 9 \& ## \& ## Define structure of your data and create serializer classes \& ## \& use Google::ProtocolBuffers; \& Google::ProtocolBuffers\->parse(" \& message Person { \& required string name = 1; \& required int32 id = 2; // Unique ID number for this person. \& optional string email = 3; \& \& enum PhoneType { \& MOBILE = 0; \& HOME = 1; \& WORK = 2; \& } \& \& message PhoneNumber { \& required string number = 1; \& optional PhoneType type = 2 [default = HOME]; \& } \& \& repeated PhoneNumber phone = 4; \& } \& ", \& {create_accessors => 1 } \& ); \& \& ## \& ## Serialize Perl structure and print it to file \& ## \& open my($fh), ">person.dat"; \& binmode $fh; \& print $fh Person\->encode({ \& name => \*(AqA.U. Thor\*(Aq, \& id => 123, \& phone => [ \& { number => 1234567890 }, \& { number => 987654321, type=>Person::PhoneType::WORK() }, \& ], \& }); \& close $fh; \& \& ## \& ## Decode data from serialized form \& ## \& my $person; \& { \& open my($fh), "decode(<$fh>); \& close $fh; \& } \& print $person\->{name}, "\en"; \& print $person\->name, "\en"; ## ditto .Ve .SH "DESCRIPTION" .IX Header "DESCRIPTION" Google Protocol Buffers is a data serialization format. It is binary (and hence compact and fast for serialization) and as extendable as \s-1XML\s0; its nearest analogues are Thrift and \s-1ASN.1.\s0 There are official mappings for \*(C+, Java and Python languages; this library is a mapping for Perl. .SH "METHODS" .IX Header "METHODS" .SS "Google::ProtocolBuffers\->parse($proto_text, \e%options)" .IX Subsection "Google::ProtocolBuffers->parse($proto_text, %options)" .SS "Google::ProtocolBuffers\->parsefile($proto_filename, \e%options)" .IX Subsection "Google::ProtocolBuffers->parsefile($proto_filename, %options)" Protocol Buffers is a typed protocol, so work with it starts with some kind of Interface Definition Language named 'proto'. For the description of the language, please see the official page () Methods 'parse' and 'parsefile' take the description of data structure as text literal or as name of the proto file correspondently. After successful compilation, Perl serializer classes are created for each message, group or enum found in proto. In case of error, these methods will die. On success, a list of names of created classes is returned. Options are given as a hash reference, the recognizable options are: .ie n .IP "include_dir => [ $dir_name ]" 4 .el .IP "include_dir => [ \f(CW$dir_name\fR ]" 4 .IX Item "include_dir => [ $dir_name ]" One proto file may include others, this option sets where to look for the included files. Multiple dirs should be specificed as an \s-1ARRAYREF.\s0 .ie n .IP "generate_code => $filename or $file_handler" 4 .el .IP "generate_code => \f(CW$filename\fR or \f(CW$file_handler\fR" 4 .IX Item "generate_code => $filename or $file_handler" Compilation of proto source is a relatively slow and memory consuming operation, it is not recommended in production environment. Instead, with this option you may specify filename or filehandle where to save Perl code of created serializer classes for future use. Example: .Sp .Vb 6 \& ## in helper script \& use Google::ProtocolBuffers; \& Google::ProtocolBuffers\->parse( \& "message Foo {optional int32 a = 1; }", \& { generate_code => \*(AqFoo.pm\*(Aq } \& ); \& \& ## then, in production code \& use Foo; \& my $str = Foo\->encode({a => 100}); .Ve .IP "create_accessors (Boolean)" 4 .IX Item "create_accessors (Boolean)" If this option is set, then result of 'decode' will be a blessed structure with accessor methods for each field, look at Class::Accessor for more info. Example: .Sp .Vb 8 \& use Google::ProtocolBuffers; \& Google::ProtocolBuffers\->parse( \& "message Foo { optional int32 id = 1; }", \& { create_accessors => 1 } \& ); \& my $foo = Foo\->decode("\ex{08}\ex{02}"); \& print $foo\->id; ## prints 2 \& $foo\->id(100); ## now it is set to 100 .Ve .IP "follow_best_practice (Boolean)" 4 .IX Item "follow_best_practice (Boolean)" This option is from Class::Accessor too; it has no effect without \&'create_accessors'. If set, names of getters (read accessors) will start with get_ and names of setter with set_: .Sp .Vb 9 \& use Google::ProtocolBuffers; \& Google::ProtocolBuffers\->parse( \& "message Foo { optional int32 id = 1; }", \& { create_accessors => 1, follow_best_practice => 1 } \& ); \& ## Class::Accessor provides a constructor too \& my $foo = Foo\->new({ id => 2 }); \& print $foo\->get_id; \& $foo\->set_id(100); .Ve .IP "simple_extensions (Boolean)" 4 .IX Item "simple_extensions (Boolean)" If this option is set, then extensions are treated as if they were regular fields in messages or groups: .Sp .Vb 10 \& use Google::ProtocolBuffers; \& use Data::Dumper; \& Google::ProtocolBuffers\->parse( \& " \& message Foo { \& optional int32 id = 1; \& extensions 10 to max; \& } \& extend Foo { \& optional string name = 10; \& } \& ", \& { simple_extensions=>1, create_accessors => 1 } \& ); \& my $foo = Foo\->decode("\ex{08}\ex{02}R\ex{03}Bob"); \& print Dumper $foo; ## { id => 2, name => \*(AqBob\*(Aq } \& print $foo\->id, "\en"; \& $foo\->name("Sponge Bob"); .Ve .Sp This option is off by default because extensions live in a separate namespace and may have the same names as fields. Compilation of such proto with \&'simple_extension' option will result in die. If the option is off, you have to use special accessors for extension fields \- setExtension and getExtension, as in \*(C+ Protocol Buffer \s-1API.\s0 Hash keys for extended fields in Plain Old Data structures will be enclosed in brackets: .Sp .Vb 10 \& use Google::ProtocolBuffers; \& use Data::Dumper; \& Google::ProtocolBuffers\->parse( \& " \& message Foo { \& optional int32 id = 1; \& extensions 10 to max; \& } \& extend Foo { \& optional string id = 10; // <\-\- id again! \& } \& ", \& { simple_extensions => 0, ## <\-\- no simple extensions \& create_accessors => 1, \& } \& ); \& my $foo = Foo\->decode("\ex{08}\ex{02}R\ex{05}Kenny"); \& print Dumper $foo; ## { id => 2, \*(Aq[id]\*(Aq => \*(AqKenny\*(Aq } \& print $foo\->id, "\en"; ## 2 \& print $foo\->getExtension(\*(Aqid\*(Aq), "\en"; ## Kenny \& $foo\->setExtension("id", \*(AqKenny McCormick\*(Aq); .Ve .IP "no_camel_case (Boolean)" 4 .IX Item "no_camel_case (Boolean)" By default, names of created Perl classes are taken from \&\*(L"camel-cased\*(R" names of proto's packages, messages, groups and enums. First characters are capitalized, all underscores are removed and the characters following them are capitalized too. An example: a fully qualified name 'package_test.Message' will result in Perl class \&'PackageTest::Message'. Option 'no_camel_case' turns name-mangling off. Names of fields, extensions and enum constants are not affected anyway. .IP "package_name (String)" 4 .IX Item "package_name (String)" Package name to be put into generated Perl code; has no effect on Perl classes names and has no effect unless 'generate_code' is also set. .SS "MessageClass\->encode($hashref)" .IX Subsection "MessageClass->encode($hashref)" This method may be called as class or instance method. 'MessageClass' must already be created by compiler. Input is a hash reference. Output is a scalar (string) with serialized data. Unknown fields in hashref are ignored. In case of errors (e.g. required field is not set and there is no default value for the required field) an exception is thrown. Examples: .PP .Vb 8 \& use Google::ProtocolBuffers; \& Google::ProtocolBuffers\->parse( \& "message Foo {optional int32 id = 1; }", \& {create_accessors => 1} \& ); \& my $string = Foo\->encode({ id => 2 }); \& my $foo = Foo\->new({ id => 2 }); \& $string = $foo\->encode; ## ditto .Ve .SS "MessageClass\->decode($scalar)" .IX Subsection "MessageClass->decode($scalar)" Class method. Input: serialized data string. Output: data object of class \&'MessageClass'. Unknown fields in serialized data are ignored. In case of errors (e.g. message is broken or partial) or data string is a wide-character (utf\-8) string, an exception is thrown. .SH "PROTO ELEMENTS" .IX Header "PROTO ELEMENTS" .SS "Enums" .IX Subsection "Enums" For each enum in proto, a Perl class will be constructed with constants for each enum value. You may import these constants via ClassName\->import(\*(L":constants\*(R") call. Please note that Perl compiler will know nothing about these constants at compile time, because this import occurs at run time, so parenthesis after constant's name are required. .PP .Vb 10 \& use Google::ProtocolBuffers; \& Google::ProtocolBuffers\->parse( \& " \& enum Foo { \& FOO = 1; \& BAR = 2; \& } \& ", \& { generate_code => \*(AqFoo.pm\*(Aq } \& ); \& print Foo::FOO(), "\en"; ## fully quailified name is fine \& Foo\->import(":constants"); \& print FOO(), "\en"; ## now FOO is defined in our namespace \& print FOO; ## <\-\- Error! FOO is bareword! .Ve .PP Or, do the import inside a \s-1BEGIN\s0 block: .PP .Vb 3 \& use Foo; ## Foo.pm was generated in previous example \& BEGIN { Foo\->import(":constants") } \& print FOO, "\en"; ## ok, Perl compiler knows about FOO here .Ve .SS "Groups" .IX Subsection "Groups" Though group are considered deprecated they are supported by Google::ProtocolBuffers. They are like nested messages, except that nested type definition and field definition go together: .PP .Vb 10 \& use Google::ProtocolBuffers; \& Google::ProtocolBuffers\->parse( \& " \& message Foo { \& optional group Bar = 1 { \& optional int32 baz = 1; \& } \& } \& ", \& { create_accessors => 1 } \& ); \& my $foo = Foo\->new; \& $foo\->Bar( Foo::Bar\->new({ baz => 2 }) ); \& print $foo\->Bar\->baz, ", ", $foo\->{Bar}\->{baz}, "\en"; # 2, 2 .Ve .SS "Default values" .IX Subsection "Default values" Proto file may specify a default value for a field. The default value is returned by accessor if there is no value for field or if this value is undefined. The default value is not accessible via plain old data hash, though. Default string values are always byte-strings, if you need wide-character (Unicode) string, use \*(L"decode_utf8\*(R" in Encode. .PP .Vb 5 \& use Google::ProtocolBuffers; \& Google::ProtocolBuffers\->parse( \& "message Foo {optional string name=1 [default=\*(AqKenny\*(Aq];} ", \& {create_accessors => 1} \& ); \& \& ## no initial value \& my $foo = Foo\->new; \& print $foo\->name(), ", ", $foo\->{name}, "\en"; # Kenny, (undef) \& \& ## some defined value \& $foo\->name(\*(AqKen\*(Aq); \& print $foo\->name(), ", ", $foo\->{name}, "\en"; # Ken, Ken \& \& ## empty, but still defined value \& $foo\->name(\*(Aq\*(Aq); \& print $foo\->name(), ", ", $foo\->{name}, "\en"; # (empty), (empty) \& \& ## undef value == default value \& $foo\->name(undef); \& print $foo\->name(), ", ", $foo\->{name}, "\en"; # Kenny, (undef) .Ve .SS "Extensions" .IX Subsection "Extensions" From the point of view of serialized data, there is no difference if a field is declared as regular field or if it is extension, as far as field number is the same. That is why there is an option 'simple_extensions' (see above) that treats extensions like regular fields. From the point of view of named accessors, however, extensions live in namespace different from namespace of fields, that's why they simple names (i.e. not fully qualified ones) may conflict. (And that's why this option is off by default). The name of extensions are obtained from their fully qualified names from which leading part, most common with the class name to be extended, is stripped. Names of hash keys enclosed in brackets; arguments to methods 'getExtension' and 'setExtension' do not. Here is the self-explanatory example to the rules: .PP .Vb 2 \& use Google::ProtocolBuffers; \& use Data::Dumper; \& \& Google::ProtocolBuffers\->parse( \& " \& package some_package; \& // message Plugh contains one regular field and three extensions \& message Plugh { \& optional int32 foo = 1; \& extensions 10 to max; \& } \& extend Plugh { \& optional int32 bar = 10; \& } \& message Thud { \& extend Plugh { \& optional int32 baz = 11; \& } \& } \& \& // Note: the official Google\*(Aqs proto compiler does not allow \& // several package declarations in a file (as of version 2.0.1). \& // To compile this example with the official protoc, put lines \& // above to some other file, and import that file here. \& package another_package; \& // import \*(Aqother_file.proto\*(Aq; \& \& extend some_package.Plugh { \& optional int32 qux = 12; \& } \& \& ", \& { create_accessors => 1 } \& ); \& \& my $plugh = SomePackage::Plugh\->decode( \& "\ex{08}\ex{01}\ex{50}\ex{02}\ex{58}\ex{03}\ex{60}\ex{04}" \& ); \& print Dumper $plugh; \& ## {foo=>1, \*(Aq[bar]\*(Aq=>2, \*(Aq[Thud.baz]\*(Aq=>3, [another_package.qux]=>4} \& \& print $plugh\->foo, "\en"; ## 1 \& print $plugh\->getExtension(\*(Aqbar\*(Aq), "\en"; ## 2 \& print $plugh\->getExtension(\*(AqThud.baz\*(Aq), "\en"; ## 3 \& print $plugh\->getExtension(\*(AqThud::baz\*(Aq), "\en"; ## ditto .Ve .PP Another point is that 'extend' block doesn't create new namespace or scope, so the following proto declaration is invalid: .PP .Vb 7 \& // proto: \& package test; \& message Foo { extensions 10 to max; } \& message Bar { extensions 10 to max; } \& extend Foo { optional int32 a = 10; } \& extend Bar { optional int32 a = 20; } // <\-\- Error: name \*(Aqa\*(Aq in package \& // \*(Aqtest\*(Aq is already used! .Ve .PP Well, extensions are the most complicated part of proto syntax, and I hope that you either got it or you don't need it. .SH "RUN-TIME MESSAGE CREATION" .IX Header "RUN-TIME MESSAGE CREATION" You don't like to mess with proto files? Structure of your data is known at run-time only? No problem, create your serializer classes at run-time too with method Google::ProtocolBuffers\->create_message('ClassName', \e@fields, \e%options); (Note: The order of field description parts is the same as in proto file. The \s-1API\s0 is going to change to accept named parameters, but backward compatibility will be preserved). .PP .Vb 2 \& use Google::ProtocolBuffers; \& use Google::ProtocolBuffers::Constants(qw/:labels :types/); \& \& ## \& ## proto: \& ## message Foo { \& ## message Bar { \& ## optional int32 a = 1 [default=12]; \& ## } \& ## required int32 id = 1; \& ## repeated Bar bars = 2; \& ## } \& ## \& Google::ProtocolBuffers\->create_message( \& \*(AqFoo::Bar\*(Aq, \& [ \& ## optional int32 a = 1 [default=12] \& [LABEL_OPTIONAL, TYPE_INT32, \*(Aqa\*(Aq, 1, \*(Aq12\*(Aq] \& ], \& { create_accessors => 1 } \& ); \& Google::ProtocolBuffers\->create_message( \& \*(AqFoo\*(Aq, \& [ \& [LABEL_REQUIRED, TYPE_INT32, \*(Aqid\*(Aq, 1], \& [LABEL_REPEATED, \*(AqFoo::Bar\*(Aq, \*(Aqbars\*(Aq, 2], \& ], \& { create_accessors => 1 } \& ); \& my $foo = Foo\->new({ id => 10 }); \& $foo\->bars( Foo::Bar\->new({a=>1}), Foo::Bar\->new({a=>2}) ); \& print $foo\->encode; .Ve .PP There are methods 'create_group' and 'create_enum' also; the following constants are exported: labels (\s-1LABEL_OPTIONAL, LABEL_OPTIONAL, LABEL_REPEATED\s0) and types (\s-1TYPE_INT32, TYPE_UINT32, TYPE_SINT32, TYPE_FIXED32, TYPE_SFIXED32, TYPE_INT64, TYPE_UINT64, TYPE_SINT64, TYPE_FIXED64, TYPE_SFIXED64,\s0 \&\s-1TYPE_BOOL, TYPE_STRING, TYPE_BYTES, TYPE_DOUBLE, TYPE_FLOAT\s0). .SH "KNOWN BUGS, LIMITATIONS AND TODOs" .IX Header "KNOWN BUGS, LIMITATIONS AND TODOs" All proto options are ignored except default values for fields; extension numbers are not checked. Unknown fields in serialized data are skipped, no stream \s-1API\s0 (encoding to/decoding from file handlers) is present. Ask for what you need most. .PP Introspection \s-1API\s0 is planned. .PP Declarations of \s-1RPC\s0 services are currently ignored, but their support is planned (btw, which Perl \s-1RPC\s0 implementation would you recommend?) .SH "SEE ALSO" .IX Header "SEE ALSO" Official page of Google's Protocol Buffers project () .PP Protobuf-PerlXS project () \- creates \s-1XS\s0 wrapper for \*(C+ classes generated by official Google's compiler protoc. You have to complile \s-1XS\s0 files every time you've changed the proto description, however, this is the fastest way to work with Protocol Buffers from Perl. .PP Protobuf-Perl project \- someday it may be part of official Google's compiler. .PP Thrift .PP \&\s-1ASN.1\s0 , \&\s-1JSON\s0 and \s-1YAML\s0 .SH "AUTHOR, ACKNOWLEDGEMENS, COPYRIGHT" .IX Header "AUTHOR, ACKNOWLEDGEMENS, COPYRIGHT" Author: Igor Gariev the \s-1CSIRT\s0 Gadgets Foundation .PP Proto grammar is based on work by Alek Storm .PP This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself, either Perl version 5.10.0 or, at your option, any later version of Perl 5 you may have available.