1/* Part of SWI-Prolog 2 3 Author: Jeffrey Rosenwald, extended by Peter Ludemann 4 E-mail: jeffrose@acm.org, peter.ludemann@gmail.com 5 WWW: http://www.swi-prolog.org 6 Copyright (c) 2010-2013, Jeffrey Rosenwald; 7 2021-2025, SWI-Prolog Solutions b.v. 8 All rights reserved. 9 10 Redistribution and use in source and binary forms, with or without 11 modification, are permitted provided that the following conditions 12 are met: 13 14 1. Redistributions of source code must retain the above copyright 15 notice, this list of conditions and the following disclaimer. 16 17 2. Redistributions in binary form must reproduce the above copyright 18 notice, this list of conditions and the following disclaimer in 19 the documentation and/or other materials provided with the 20 distribution. 21 22 THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 23 "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 24 LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS 25 FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE 26 COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, 27 INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, 28 BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; 29 LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 30 CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 31 LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN 32 ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE 33 POSSIBILITY OF SUCH DAMAGE. 34*/ 35 36:- module(protobufs, 37 [ protobuf_message/2, % ?Template ?Codes 38 protobuf_message/3, % ?Template ?Codes ?Rest 39 protobuf_parse_from_codes/3, % +WireCodes, +MessageType, -Term 40 protobuf_serialize_to_codes/3, % +Term, +MessageType, -WireCodes 41 protobuf_field_is_map/2, % +MessageType, +FieldName 42 protobuf_map_pairs/3 % ?ProtobufTermList, ?DictTag, ?Pairs 43 44 % TODO: Restore the following to the public interface, if 45 % someone needs them. For now, the tests directly specify 46 % them using, e.g. protobufs:uint32_codes(..., ...). 47 % 48 % protobuf_segment_message/2, % ?Segments ?Codes 49 % protobuf_segment_convert/2, % +Form1 ?Form2 50 % uint32_codes/2, 51 % int32_codes/2, 52 % float32_codes/2, 53 % uint64_codes/2, 54 % int64_codes/2, 55 % float64_codes/2, 56 % int64_zigzag/2, 57 % uint32_int32/2, 58 % uint64_int64/2, 59 % uint32_codes_when/2, 60 % int32_codes_when/2, % TODO: unused 61 % float32_codes_when/2, 62 % uint64_codes_when/2, 63 % int64_codes_when/2, % TODO: unused 64 % float64_codes_when/2, 65 % int64_zigzag_when/2, 66 % uint32_int32_when/2, 67 % uint64_int64_when/2, 68 % int64_float64_when/2, 69 % int32_float32_when/2, 70 % protobuf_var_int//1, 71 % protobuf_tag_type//2 72 ]). 73 74:- use_module(library(apply_macros)). % autoload(library(apply), [maplist/3, foldl/4]). 75:- autoload(library(error), [must_be/2, domain_error/2, existence_error/2]). 76:- autoload(library(lists), [append/3]). 77:- autoload(library(utf8), [utf8_codes//1]). 78:- autoload(library(dif), [dif/2]). 79:- autoload(library(dcg/high_order), [sequence//2]). 80:- autoload(library(when), [when/2]). 81:- use_module(library(debug), [assertion/1]). % TODO: remove 82 83:- set_prolog_flag(optimise, true). % For arithmetic using is/2.
163:- use_foreign_library(foreign(protobufs)).MessageType), creating a Prolog term.
Protoc must have been run (with the --swipl_out= option and the resulting
top-level _pb.pl file loaded. For more details, see the "protoc" section of the
overview documentation.
Fails if the message can't be parsed or if the appropriate meta-data from protoc
hasn't been loaded.
All fields that are omitted from the WireCodes are set to their
default values (typically the empty string or 0, depending on the
type; or [] for repeated groups). There is no way of testing
whether a value was specified in WireCodes or given its default
value (that is, there is no equivalent of the Python
implementation's =HasField`). Optional embedded messages and groups
do not have any default value -- you must check their existence by
using get_dict/3 or similar. If a field is part of a "oneof" set,
then none of the other fields is set. You can determine which field
had a value by using get_dict/3.
220protobuf_parse_from_codes(WireCodes, MessageType0, Term) :- 221 verify_version, 222 must_be(ground, MessageType0), 223 ( proto_meta_normalize(MessageType0, MessageType) 224 -> true 225 ; existence_error(protobuf_package, MessageType0) 226 ), 227 protobuf_segment_message(Segments, WireCodes), 228 % protobuf_segment_message/2 can leave choicepoints, backtracking 229 % through all the possibilities would have combinatoric explosion; 230 % instead use segment_to_term/3 call protobuf_segment_convert/2 to 231 % change segments that were guessed incorrectly. 232 !, 233 maplist(segment_to_term(MessageType), Segments, MsgFields), 234 !, % TODO: remove 235 combine_fields(MsgFields, MessageType{}, Term), 236 !. % TODO: remove? - but proto_meta might have left choicepoints if loaded twice 237 238verify_version :- 239 ( protoc_gen_swipl_version(Module, Version), 240 Version @< [0,9,1] % This must be sync-ed with changes to protoc-gen-swipl 241 -> throw(error(version_error(Module-Version), _)) 242 ; true 243 ).
MessageType).
Protoc must have been run (with the --swipl_out= option and the resulting
top-level _pb.pl file loaded. For more details, see the "protoc" section of the
overview documentation.
Fails if the term isn't of an appropriate form or if the appropriate
meta-data from protoc hasn't been loaded, or if a field name is incorrect
(and therefore nothing in the meta-data matches it).
278protobuf_serialize_to_codes(Term, MessageType0, WireCodes) :- 279 verify_version, 280 must_be(ground, MessageType0), 281 ( proto_meta_normalize(MessageType0, MessageType) 282 -> true 283 ; existence_error(protobuf_package, MessageType0) 284 ), 285 term_to_segments(Term, MessageType, Segments), 286 !, % TODO: remove 287 protobuf_segment_message(Segments, WireCodes), 288 !. % TODO: remove? - but proto_meta might have left choicepoints if loaded twice 289 290% 291% Map wire type (atom) to its encoding (an int) 292% 293wire_type(varint, 0). % for int32, int64, uint32, uint64, sint32, sint64, bool, enum 294wire_type(fixed64, 1). % for fixed64, sfixed64, double 295wire_type(length_delimited, 2). % for string, bytes, embedded messages, packed repeated fields 296wire_type(start_group, 3). % for groups (deprecated) 297wire_type(end_group, 4). % for groups (deprecated) 298wire_type(fixed32, 5). % for fixed32, sfixed32, float 299 300% 301% basic wire-type processing handled by C-support code in DCG-form 302% 303 304fixed_uint32(X, [A0, A1, A2, A3 | Rest], Rest) :- 305 uint32_codes_when(X, [A0, A1, A2, A3]). 306/* equivalent to: 307fixed_uint32_(X) --> 308 [ A0,A1,A2,A3 ], 309 { uint32_codes_when(X, [A0,A1,A2,A3]) }. 310*/ 311 312fixed_uint64(X, [A0, A1, A2, A3, A4, A5, A6, A7 | Rest], Rest) :- 313 uint64_codes_when(X, [A0, A1, A2, A3, A4, A5, A6, A7]). 314 315fixed_float64(X, [A0, A1, A2, A3, A4, A5, A6, A7 | Rest], Rest) :- 316 float64_codes_when(X, [A0, A1, A2, A3, A4, A5, A6, A7]). 317 318fixed_float32(X, [A0, A1, A2, A3 | Rest], Rest) :- 319 float32_codes_when(X, [A0, A1, A2, A3]). 320 321% 322% Start of the DCG 323% 324 325code_string(N, Codes, Rest, Rest1) :- 326 length(Codes, N), 327 append(Codes, Rest1, Rest), 328 !. 329/* 330code_string(N, Codes) --> 331 { length(Codes, N) }, 332 Codes, !. 333*/ 334 335% 336% deal with Google's method of packing unsigned integers in variable 337% length, modulo 128 strings. 338% 339% protobuf_var_int//1 and protobuf_tag_type//2 productions were rewritten in straight 340% Prolog for speed's sake. 341%
A is negative.
This is a low-level predicate; normally, you should use
template_message/2 and the appropriate template term.
e.g. phrase(protobuf_var_int(300), S) => S = [172,2]
phrase(protobuf_var_int(A), [172,2]) -> A = 300351protobuf_var_int(A, [A | Rest], Rest) :- 352 A < 128, 353 !. 354protobuf_var_int(X, [A | Rest], Rest1) :- 355 nonvar(X), 356 X1 is X >> 7, 357 A is 128 + (X /\ 0x7f), 358 protobuf_var_int(X1, Rest, Rest1), 359 !. 360protobuf_var_int(X, [A | Rest], Rest1) :- 361 protobuf_var_int(X1, Rest, Rest1), 362 X is (X1 << 7) + A - 128, 363 !.
372protobuf_tag_type(Tag, WireType, Rest, Rest1) :- 373 nonvar(Tag), nonvar(WireType), 374 wire_type(WireType, WireTypeEncoding), 375 A is Tag << 3 \/ WireTypeEncoding, 376 protobuf_var_int(A, Rest, Rest1), 377 !. 378protobuf_tag_type(Tag, WireType, Rest, Rest1) :- 379 protobuf_var_int(A, Rest, Rest1), 380 WireTypeEncoding is A /\ 0x07, 381 wire_type(WireType, WireTypeEncoding), 382 Tag is A >> 3.
390prolog_type(Tag, double) --> protobuf_tag_type(Tag, fixed64). 391prolog_type(Tag, integer64) --> protobuf_tag_type(Tag, fixed64). 392prolog_type(Tag, unsigned64) --> protobuf_tag_type(Tag, fixed64). 393prolog_type(Tag, float) --> protobuf_tag_type(Tag, fixed32). 394prolog_type(Tag, integer32) --> protobuf_tag_type(Tag, fixed32). 395prolog_type(Tag, unsigned32) --> protobuf_tag_type(Tag, fixed32). 396prolog_type(Tag, integer) --> protobuf_tag_type(Tag, varint). 397prolog_type(Tag, unsigned) --> protobuf_tag_type(Tag, varint). 398prolog_type(Tag, signed32) --> protobuf_tag_type(Tag, varint). 399prolog_type(Tag, signed64) --> protobuf_tag_type(Tag, varint). 400prolog_type(Tag, boolean) --> protobuf_tag_type(Tag, varint). 401prolog_type(Tag, enum) --> protobuf_tag_type(Tag, varint). 402prolog_type(Tag, atom) --> protobuf_tag_type(Tag, length_delimited). 403prolog_type(Tag, codes) --> protobuf_tag_type(Tag, length_delimited). 404prolog_type(Tag, utf8_codes) --> protobuf_tag_type(Tag, length_delimited). 405prolog_type(Tag, string) --> protobuf_tag_type(Tag, length_delimited). 406prolog_type(Tag, embedded) --> protobuf_tag_type(Tag, length_delimited). 407prolog_type(Tag, packed) --> protobuf_tag_type(Tag, length_delimited). 408 409% 410% The protobuf-2.1.0 grammar allows negative values in enums. 411% But they are encoded as unsigned in the golden message. 412% As such, they use the maximum length of a varint, so it is 413% recommended that they be non-negative. However, that's controlled 414% by the =|.proto|= file. 415% 416:- meta_predicate enumeration(,,). 417 418enumeration(Type) --> 419 { call(Type, Value) }, 420 payload(signed64, Value).
Payload, according to PrologType
TODO: payload//2 "mode" is sometimes module-sensitive, sometimes not.
payload(enum, A)// has A as a callable
all other uses of payload//2, the 2nd arg is not callable.
vector_demo.pl, which defines commands/2)430payload(enum, Payload) --> 431 enumeration(Payload). 432payload(double, Payload) --> 433 fixed_float64(Payload). 434payload(integer64, Payload) --> 435 { uint64_int64_when(Payload0, Payload) }, 436 fixed_uint64(Payload0). 437payload(unsigned64, Payload) --> 438 fixed_uint64(Payload). 439payload(float, Payload) --> 440 fixed_float32(Payload). 441payload(integer32, Payload) --> 442 { uint32_int32_when(Payload0, Payload) }, 443 fixed_uint32(Payload0). 444payload(unsigned32, Payload) --> 445 fixed_uint32(Payload). 446payload(integer, Payload) --> 447 { nonvar(Payload), int64_zigzag(Payload, X) }, % TODO: int64_zigzag_when/2 448 !, 449 protobuf_var_int(X). 450payload(integer, Payload) --> 451 protobuf_var_int(X), 452 { int64_zigzag(Payload, X) }. % TODO: int64_zigzag_when/2 453payload(unsigned, Payload) --> 454 protobuf_var_int(Payload), 455 { Payload >= 0 }. 456payload(signed32, Payload) --> % signed32 is not defined by prolog_type//2 457 % for wire-stream compatibility reasons. 458 % signed32 ought to write 5 bytes for negative numbers, but both 459 % the C++ and Python implementations write 10 bytes. For 460 % wire-stream compatibility, we follow C++ and Python, even though 461 % protoc decode appears to work just fine with 5 bytes -- 462 % presumably there are some issues with decoding 5 bytes and 463 % getting the sign extension correct with some 32/64-bit integer 464 % models. See CodedOutputStream::WriteVarint32SignExtended(int32 465 % value) in google/protobuf/io/coded_stream.h. 466 payload(signed64, Payload). 467payload(signed64, Payload) --> 468 % protobuf_var_int//1 cannot handle negative numbers (note that 469 % zig-zag encoding always results in a positive number), so 470 % compute the 64-bit 2s complement, which is what is produced 471 % form C++ and Python. 472 { nonvar(Payload) }, 473 !, 474 { uint64_int64(X, Payload) }, % TODO: uint64_int64_when 475 protobuf_var_int(X). 476payload(signed64, Payload) --> 477 % See comment in previous clause about negative numbers. 478 protobuf_var_int(X), 479 { uint64_int64(X, Payload) }. % TODO: uint64_int64_when 480payload(codes, Payload) --> 481 { nonvar(Payload), 482 !, 483 length(Payload, Len) 484 }, 485 protobuf_var_int(Len), 486 code_string(Len, Payload). 487payload(codes, Payload) --> 488 protobuf_var_int(Len), 489 code_string(Len, Payload). 490payload(utf8_codes, Payload) --> 491 { nonvar(Payload), % TODO: use freeze/2 or when/2 492 !, 493 phrase(utf8_codes(Payload), B) 494 }, 495 payload(codes, B). 496payload(utf8_codes, Payload) --> 497 payload(codes, B), 498 { phrase(utf8_codes(Payload), B) }. 499payload(atom, Payload) --> 500 { nonvar(Payload), 501 atom_codes(Payload, Codes) 502 }, 503 payload(utf8_codes, Codes), 504 !. 505payload(atom, Payload) --> 506 payload(utf8_codes, Codes), 507 { atom_codes(Payload, Codes) }. 508payload(boolean, true) --> 509 payload(unsigned, 1). 510payload(boolean, false) --> 511 payload(unsigned, 0). 512payload(string, Payload) --> 513 { nonvar(Payload) 514 -> string_codes(Payload, Codes) 515 ; true 516 }, 517 % string_codes produces a list of unicode, not bytes 518 payload(utf8_codes, Codes), 519 { string_codes(Payload, Codes) }. 520payload(embedded, protobuf(PayloadSeq)) --> 521 { ground(PayloadSeq), 522 phrase(protobuf(PayloadSeq), Codes) 523 }, 524 payload(codes, Codes), 525 !. 526payload(embedded, protobuf(PayloadSeq)) --> 527 payload(codes, Codes), 528 { phrase(protobuf(PayloadSeq), Codes) }. 529payload(packed, TypedPayloadSeq) --> 530 { TypedPayloadSeq =.. [PrologType, PayloadSeq], % TypedPayloadSeq = PrologType(PayloadSeq) 531 ground(PayloadSeq), 532 phrase(packed_payload(PrologType, PayloadSeq), Codes) 533 }, 534 payload(codes, Codes), 535 !. 536payload(packed, enum(EnumSeq)) --> 537 !, 538 % TODO: combine with next clause 539 % TODO: replace =.. with a predicate that gives all the possibilities - see detag/6. 540 { EnumSeq =.. [ Enum, Values ] }, % EnumSeq = Enum(Values) 541 payload(codes, Codes), 542 { phrase(packed_enum(Enum, Values), Codes) }. 543payload(packed, TypedPayloadSeq) --> 544 payload(codes, Codes), 545 % TODO: replace =.. with a predicate that gives all the possibilities - see detag/6. 546 { TypedPayloadSeq =.. [PrologType, PayloadSeq] }, % TypedPayloadSeq = PrologType(PayloadSeq) 547 { phrase(packed_payload(PrologType, PayloadSeq), Codes) }. 548 549packed_payload(enum, EnumSeq) --> 550 { ground(EnumSeq) }, !, 551 { EnumSeq =.. [EnumType, Values] }, % EnumSeq = EnumType(Values) 552 packed_enum(EnumType, Values). 553packed_payload(PrologType, PayloadSeq) --> 554 sequence_payload(PrologType, PayloadSeq). 555 556% sequence_payload//2 (because sequence//2 isn't compile-time expanded) 557sequence_payload(PrologType, PayloadSeq) --> 558 sequence_payload_(PayloadSeq, PrologType). 559 560sequence_payload_([], _PrologType) --> [ ]. 561sequence_payload_([Payload|PayloadSeq], PrologType) --> 562 payload(PrologType, Payload), 563 sequence_payload_(PayloadSeq, PrologType). 564 565packed_enum(Enum, [ A | As ]) --> 566 % TODO: replace =.. with a predicate that gives all the possibilities - see detag/6. 567 { E =.. [Enum, A] }, 568 payload(enum, E), 569 packed_enum(Enum, As). 570packed_enum(_, []) --> [ ]. 571 572start_group(Tag) --> protobuf_tag_type(Tag, start_group). 573 574end_group(Tag) --> protobuf_tag_type(Tag, end_group). 575% 576% 577nothing([]) --> [], !. 578 579protobuf([Field | Fields]) --> 580 % TODO: don't use =.. -- move logic to single_message 581 ( { Field = repeated_embedded(Tag, protobuf(EmbeddedFields), Items) } 582 -> repeated_embedded_messages(Tag, EmbeddedFields, Items) 583 ; { Field =.. [ PrologType, Tag, Payload] }, % Field = PrologType(Tag, Payload) 584 single_message(PrologType, Tag, Payload), 585 ( protobuf(Fields) 586 ; nothing(Fields) 587 ) 588 ), 589 !. 590 591repeated_message(repeated_enum, Tag, Type, [A | B]) --> 592 % TODO: replace =.. with a predicate that gives all the possibilities - see detag/6. 593 { TypedPayload =.. [Type, A] }, % TypedPayload = Type(A) 594 single_message(enum, Tag, TypedPayload), 595 ( repeated_message(repeated_enum, Tag, Type, B) 596 ; nothing(B) 597 ). 598repeated_message(Type, Tag, [A | B]) --> 599 { Type \= repeated_enum }, 600 single_message(Type, Tag, A), 601 repeated_message(Type, Tag, B). 602repeated_message(_Type, _Tag, A) --> 603 nothing(A). 604 605repeated_embedded_messages(Tag, EmbeddedFields, [protobuf(A) | B]) --> 606 { copy_term(EmbeddedFields, A) }, 607 single_message(embedded, Tag, protobuf(A)), !, 608 repeated_embedded_messages(Tag, EmbeddedFields, B). 609repeated_embedded_messages(_Tag, _EmbeddedFields, []) --> 610 [ ].
protobuf([...]).
The PrologType, Tag, Payload are from Field =.. [PrologType, Tag, Payload]
in the caller616single_message(repeated, Tag, enum(EnumSeq)) --> 617 !, 618 { EnumSeq =.. [EnumType, Values] }, % EnumSeq = EnumType(Values) 619 repeated_message(repeated_enum, Tag, EnumType, Values). 620single_message(repeated, Tag, Payload) --> 621 !, 622 % TODO: replace =.. with a predicate that gives all the possibilities - see detag/6. 623 { Payload =.. [PrologType, A] }, % Payload = PrologType(A) 624 { PrologType \= enum }, 625 repeated_message(PrologType, Tag, A). 626single_message(group, Tag, A) --> 627 !, 628 start_group(Tag), 629 protobuf(A), 630 end_group(Tag). 631single_message(PrologType, Tag, Payload) --> 632 { PrologType \= repeated, PrologType \= group }, 633 prolog_type(Tag, PrologType), 634 payload(PrologType, Payload).
659protobuf_message(protobuf(TemplateList), WireStream) :- 660 must_be(list, TemplateList), 661 phrase(protobuf(TemplateList), WireStream), 662 !. 663 664protobuf_message(protobuf(TemplateList), WireStream, Residue) :- 665 must_be(list, TemplateList), 666 phrase(protobuf(TemplateList), WireStream, Residue).
.proto description, similar to
the processing done by protoc --decode_raw. This means that
field names aren't shown, only field numbers.
For unmarshalling, a simple heuristic is used on length-delimited segments: first interpret it as a message; if that fails, try to interpret as a UTF8 string; otherwise, leave it as a "blob" (if the heuristic was wrong, you can convert to a string or a blob by using protobuf_segment_convert/2). 32-bit and 64-bit numbers are left as codes because they could be either integers or floating point (use int32_codes_when/2, float32_codes_when/2, int64_codes_when/2, uint32_codes_when/2, uint64_codes_when/2, float64_codes_when/2 as appropriate); variable-length numbers ("varint" in the Protocol Buffers encoding documentation), might require "zigzag" conversion, int64_zigzag_when/2.
For marshalling, use the predicates int32_codes_when/2, float32_codes_when/2, int64_codes_when/2, uint32_codes_when/2, uint64_codes_when/2, float64_codes_when/2, int64_zigzag_when/2 to put integer and floating point values into the appropriate form.
739protobuf_segment_message(Segments, WireStream) :- 740 phrase(segment_message(Segments), WireStream). 741 742segment_message(Segments) --> 743 sequence_segment(Segments). 744 745% sequence_segment//1 (because sequence//2 isn't compile-time expanded) 746sequence_segment([]) --> [ ]. 747sequence_segment([Segment|Segments]) --> 748 segment(Segment), 749 sequence_segment(Segments). 750 751segment(Segment) --> 752 { nonvar(Segment) }, 753 !, 754 % repeated(List) can be created by field_segment_scalar_or_repeated/7 755 ( { Segment = repeated(Segments) } 756 -> sequence_segment(Segments) 757 ; { segment_type_tag(Segment, Type, Tag) }, 758 protobuf_tag_type(Tag, Type), 759 segment(Type, Tag, Segment) 760 ). 761segment(Segment) --> 762 % { var(Segment) }, 763 protobuf_tag_type(Tag, Type), 764 segment(Type, Tag, Segment). 765 766segment(varint, Tag, varint(Tag,Value)) --> 767 protobuf_var_int(Value). 768segment(fixed64, Tag, fixed64(Tag, Int64)) --> 769 payload(integer64, Int64). 770segment(fixed32, Tag, fixed32(Tag, Int32)) --> 771 payload(integer32, Int32). 772segment(start_group, Tag, group(Tag, Segments)) --> 773 segment_message(Segments), 774 protobuf_tag_type(Tag, end_group). 775segment(length_delimited, Tag, Result) --> 776 segment_length_delimited(Tag, Result). 777 778segment_length_delimited(Tag, Result) --> 779 { nonvar(Result) }, 780 !, 781 { length_delimited_segment(Result, Tag, Codes) }, 782 { length(Codes, CodesLen) }, 783 protobuf_var_int(CodesLen), 784 code_string(CodesLen, Codes). 785segment_length_delimited(Tag, Result) --> 786 % { var(Result) }, 787 protobuf_var_int(CodesLen), 788 code_string(CodesLen, Codes), 789 { length_delimited_segment(Result, Tag, Codes) }. 790 791length_delimited_segment(message(Tag,Segments), Tag, Codes) :- 792 protobuf_segment_message(Segments, Codes). 793length_delimited_segment(group(Tag,Segments), Tag, Codes) :- 794 phrase(segment_group(Tag, Segments), Codes). 795length_delimited_segment(string(Tag,String), Tag, Codes) :- 796 ( nonvar(String) 797 -> string_codes(String, StringCodes), 798 phrase(utf8_codes(StringCodes), Codes) 799 ; phrase(utf8_codes(StringCodes), Codes), 800 string_codes(String, StringCodes) 801 ). 802length_delimited_segment(packed(Tag,Payload), Tag, Codes) :- 803 % We don't know the type of the fields, so we try the 3 804 % possibilities. This has a problem: an even number of fixed32 805 % items can't be distinguished from half the number of fixed64 806 % items; but it's all we can do. The good news is that usually 807 % varint (possibly with zig-zag encoding) is more common because 808 % it's more compact (I don't know whether 32-bit or 64-bit is more 809 % common for floating point). 810 packed_option(Type, Items, Payload), 811 phrase(sequence_payload(Type, Items), Codes). 812length_delimited_segment(length_delimited(Tag,Codes), Tag, Codes). 813 814segment_group(Tag, Segments) --> 815 start_group(Tag), 816 segment_message(Segments), 817 end_group(Tag). 818 819% See also prolog_type//2. Note that this doesn't handle repeated(List), 820% which is used internally (see field_segment_scalar_or_repeated/7). 821segment_type_tag(varint(Tag,_Value), varint, Tag). 822segment_type_tag(fixed64(Tag,_Value), fixed64, Tag). 823segment_type_tag(group(Tag,_Segments), start_group, Tag). 824segment_type_tag(fixed32(Tag,_Value), fixed32, Tag). 825segment_type_tag(length_delimited(Tag,_Codes), length_delimited, Tag). 826segment_type_tag(message(Tag,_Segments), length_delimited, Tag). 827segment_type_tag(packed(Tag,_Payload), length_delimited, Tag). 828segment_type_tag(string(Tag,_String), length_delimited, Tag).
Compound or the form Name(Tag,Value) and create a
new CompoundWithList that replaces Value with List. This is
used by packed_list/2 to transform [varint(1,0),varint(1,1)] to
varint(1,[0,1]).
Some of Compound items are impossible for packed with the
current protobuf spec, but they don't do any harm.
838detag(varint(Tag,Value), varint, Tag, Value, List, varint(List)). 839detag(fixed64(Tag,Value), fixed64, Tag, Value, List, fixed64(List)). 840detag(fixed32(Tag,Value), fixed32, Tag, Value, List, fixed32(List)). 841detag(length_delimited(Tag,Codes), length_delimited, Tag, Codes, List, length_delimited(List)). 842detag(message(Tag,Segments), message, Tag, Segments, List, message(List)). 843detag(packed(Tag,Payload), packed, Tag, Payload, List, packed(List)). % TODO: delete? 844detag(string(Tag,String), string, Tag, String, List, string(List)). 845 846% See also prolog_type//2, but pick only one for each wirestream type 847% For varint(Items), use one that doesn't do zigzag 848packed_option(integer64, Items, fixed64(Items)). 849packed_option(integer32, Items, fixed32(Items)). 850packed_option(unsigned, Items, varint(Items)). 851% packed_option(integer, Items, varint(Items)). 852% packed_option(double, Items, fixed64(Items)). 853% packed_option(float, Items, fixed32(Items)). 854% packed_option(signed64, Items, varint(Items)). 855% packed_option(boolean, Items, varint(Items)). 856% packed_option(enum, Items, varint(Items)).
Form1 is converted back to the original wire stream, then the
predicate non-deterimisticly attempts to convert the wire stream to
a string or length_delimited term (or both: the lattter
always succeeds).
The possible conversions are:
message(Tag,Segments) => string(Tag,String)
message(Tag,Segments) => length_delimited(Tag,Codes)
string(Tag,String) => length_delimited(Tag,Codes)
length_delimited(Tag,Codes) => length_delimited(Tag,Codes)
Note that for fixed32, fixed64, only the signed integer forms are given; if you want the floating point forms, then you need to do use int64_float64_when/2 and int32_float32_when/2.
For example:
?- protobuf_segment_convert(
message(10,[fixed64(13,7309475598860382318)]),
string(10,"inputType")).
?- protobuf_segment_convert(
message(10,[fixed64(13,7309475598860382318)]),
length_delimited(10,[105,110,112,117,116,84,121,112,101])).
?- protobuf_segment_convert(
string(10, "inputType"),
length_delimited(10,[105,110,112,117,116,84,121,112,101])).
?- forall(protobuf_segment_convert(string(1999,"\x1\\x0\\x0\\x0\\x2\\x0\\x0\\x0\"),Z), writeln(Z)).
string(1999, )
packed(1999,fixed64([8589934593]))
packed(1999,fixed32([1,2]))
packed(1999,varint([1,0,0,0,2,0,0,0]))
length_delimited(1999,[1,0,0,0,2,0,0,0])
These come from:
Codes = [82,9,105,110,112,117,116,84,121,112,101], protobuf_message(protobuf([embedded(T1, protobuf([integer64(T2, I)]))]), Codes), protobuf_message(protobuf([string(T,S)]), Codes). T = 10, T1 = 10, T2 = 13, I = 7309475598860382318, S = "inputType".
915protobuf_segment_convert(Form, Form). % for efficiency, don't generate codes 916protobuf_segment_convert(Form1, Form2) :- 917 dif(Form1, Form2), % Form1=Form2 handled by first clause 918 protobuf_segment_message([Form1], WireCodes), 919 phrase(tag_and_codes(Tag, Codes), WireCodes), 920 length_delimited_segment(Form2, Tag, Codes). 921 922tag_and_codes(Tag, Codes) --> 923 protobuf_tag_type(Tag, length_delimited), 924 payload(codes, Codes). 925 926%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 927% Documention of the foreign predicates, which are wrapped and exported.
This predicate delays until either Uint32 or Codes is
sufficiently instantiated.
There is also a non-delayed uint32_codes/2
SWI-Prolog doesn't have a 32-bit integer type, so 32-bit integer is simulated by doing a range check.
947uint32_codes_when(Uint32, Codes) :-
948 when((nonvar(Uint32) ; nonvar(Codes)), uint32_codes(Uint32, Codes)).
This predicate delays until either Int32 or Codes is
sufficiently instantiated.
There is also a non-delayed int32_codes/2
SWI-Prolog doesn't have a 32-bit integer type, so 32-bit integer is simulated by doing a range check.
968int32_codes_when(Int32, Codes) :- % TODO: unused
969 when((nonvar(Int32) ; nonvar(Codes)), int32_codes(Int32, Codes)).
This predicate delays until either Value or Codes is
sufficiently instantiated.
There is also a non-delayed float32_codes/2
983float32_codes_when(Value, Codes) :-
984 when((nonvar(Value) ; nonvar(Codes)), float32_codes(Value, Codes)).SWI-Prolog allows integer values greater than 64 bits, so a range check is done.
This predicate delays until either Uint64 or Codes is
sufficiently instantiated.
There is also a non-delayed uint64_codes/2
999% 1000% @param Uint64 an unsigned integer 1001% @param Codes a list of 8 integers (codes) 1002% 1003% @error Type,Domain if =Uint64= or =Codes= are of the wrong 1004% type or out of range. 1005uint64_codes_when(Uint64, Codes) :- 1006 when((nonvar(Uint64) ; nonvar(Codes)), uint64_codes(Uint64, Codes)).
SWI-Prolog allows integer values greater than 64 bits, so a range check is done.
This predicate delays until either Int64 or Codes is
sufficiently instantiated.
There is also a non-delayed int64_codes/2
1021% 1022% @param Int64 an unsigned integer 1023% @param Codes a list of 8 integers (codes) 1024% 1025% @error Type,Domain if =Int64= or =Codes= are of the wrong 1026% type or out of range. 1027int64_codes_when(Int64, Codes) :- % TODO: unused 1028 when((nonvar(Int64) ; nonvar(Codes)), int64_codes(Int64, Codes)).
This predicate delays until either Value or Codes is
sufficiently instantiated.
There is also a non-delayed float64_codes/2
1047float64_codes_when(Value, Codes) :-
1048 when((nonvar(Value) ; nonvar(Codes)), float64_codes(Value, Codes)).sint32 and sint64 types. This is a
low-level predicate; normally, you should use template_message/2 and
the appropriate template term.
SWI-Prolog allows integer values greater than 64 bits, so a range check is done.
This predicate delays until either Original or Encoded is
sufficiently instantiated.
There is also a non-delayed int64_zigzag/2
1073int64_zigzag_when(Original, Encoded) :-
1074 when((nonvar(Original) ; nonvar(Encoded)), int64_zigzag(Original, Encoded)).uint64_int64(0xffffffffffffffff,-1).
This predicate delays until either Uint64 or Int64 is
sufficiently instantiated.
There is also a non-delayed uint64_int64/2
1092uint64_int64_when(Uint64, Int64) :- 1093 when((nonvar(Uint64) ; nonvar(Int64)), uint64_int64(Uint64, Int64)). 1094 1095% Reversed argument ordering for maplist/3 1096int64_uint64_when(Int64, Uint64) :- 1097 uint64_int64_when(Uint64, Int64).
This predicate delays until either Uint32 or Int32 is
sufficiently instantiated.
There is also a non-delayed uint32_int32/2
1114uint32_int32_when(Uint32, Int32) :- 1115 when((nonvar(Uint32) ; nonvar(Int32)), uint32_int32(Uint32, Int32)). 1116 1117% Reversed argument ordering for maplist/3 1118int32_uint32_when(Int32, Uint32) :- 1119 1120 uint32_int32_when(Uint32, Int32).
int64_float64(3ff0000000000000,1.0).
This predicate delays until either Int64 or Float64 is
sufficiently instantiated.
There is also a non-delayed uint64_int64/2
1138int64_float64_when(Int64, Float64) :-
1139 when((nonvar(Int64) ; nonvar(Float64)), int64_float64(Int64, Float64)).int32_float32(0x3f800000,1.0).
This predicate delays until either Int32 or Float32 is
sufficiently instantiated.
There is also a non-delayed uint32_int32/2
1157int32_float32_when(Int32, Float32) :- 1158 when((nonvar(Int32) ; nonvar(Float32)), int32_float32(Int32, Float32)). 1159 1160 1161%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 1162% 1163% Use protobufs meta-data (see the section on protoc in the "overview" documentation). 1164 1165% The protoc plugin generates the following facts (all starting with "proto_meta_"). 1166% The are documented in protoc-gen-swipl and in the overview section. 1167 1168:- multifile 1169 proto_meta_normalize/2, % (Unnormalized, Normalized) 1170 proto_meta_package/3, % (Package, FileName, Options) 1171 proto_meta_message_type/3, % (Fqn, Package, Name) 1172 proto_meta_message_type_map_entry/1, % (Fqn) 1173 proto_meta_field_name/4, % (Fqn, FieldNumber, FieldName, FqnName) 1174 proto_meta_field_json_name/2, % (FqnName, JsonName) 1175 proto_meta_field_label/2, % (FqnName, LabelRepeatOptional) % LABEL_OPTIONAL, LABEL_REQUIRED, LABEL_REPEATED 1176 proto_meta_field_type/2, % (FqnName, Type) % TYPE_INT32, TYPE_MESSAGE, etc 1177 proto_meta_field_type_name/2, % (FqnName, TypeName) 1178 proto_meta_field_default_value/2, % (FqnName, DefaultValue) 1179 proto_meta_field_option_packed/1, % (FqnName) 1180 proto_meta_enum_type/3, % (FqnName, Fqn, Name) 1181 proto_meta_enum_value/3, % (FqnName, Name, Number) 1182 proto_meta_field_oneof_index/2, % (FqnName, Index) 1183 proto_meta_oneof/3. % (FqnName, Index, Name) 1184 1185proto_meta_enum_value_when(ContextType, EnumValue, IntValue) :- 1186 when((nonvar(EnumValue) ; nonvar(IntValue)), 1187 proto_meta_enum_value_(ContextType, EnumValue, IntValue)). 1188 1189proto_meta_enum_value_(ContextType, EnumValue, IntValue) :- 1190 ( proto_meta_enum_value(ContextType, EnumValue, IntValue) 1191 -> true 1192 ; existence_error(ContextType, EnumValue-IntValue) 1193 ). 1194 1195:- det(segment_to_term/3).
TODO: if performance is an issue, this code can be combined with
protobuf_segment_message/2 (and thereby avoid the use of protobuf_segment_convert/2)1201segment_to_term(ContextType0, Segment, FieldAndValue) => 1202 segment_type_tag(Segment, _, Tag), 1203 field_and_type(ContextType0, Tag, FieldName, _FqnName, ContextType, RepeatOptional, Type), 1204 ( RepeatOptional = repeat_packed 1205 -> convert_segment_packed(Type, ContextType, Tag, Segment, Value) 1206 ; convert_segment(Type, ContextType, Tag, Segment, Value) 1207 ), 1208 !, % TODO: get rid of this? 1209 FieldAndValue = field_and_value(FieldName,RepeatOptional,Value). 1210 1211% :- det(convert_segment_packed/5). % TODO: "succeeded with a choicepoint" 1212%! convert_segment_packed(+Type:atom, +ContextType:atom, +Tag:atom, ?Segment, ?Values) is det. 1213% Reversible on =Segment=, =Values=. 1214% 1215% TODO: these are very similar to convert_segment - can they be combined? 1216 1217convert_segment_packed('TYPE_DOUBLE', _ContextType, Tag, Segment0, Values) => 1218 freeze(Segment0, protobuf_segment_convert(Segment0, packed(Tag, fixed64(Values0)))), 1219 maplist(int64_float64_when, Values0, Values), !. 1220convert_segment_packed('TYPE_FLOAT', _ContextType, Tag, Segment0, Values) => 1221 freeze(Segment0, protobuf_segment_convert(Segment0, packed(Tag, fixed32(Values0)))), 1222 maplist(int32_float32_when, Values0, Values), !. 1223convert_segment_packed('TYPE_INT64', _ContextType, Tag, Segment0, Values) => 1224 freeze(Segment0, protobuf_segment_convert(Segment0, packed(Tag, varint(Values0)))), 1225 maplist(uint64_int64_when, Values0, Values). 1226convert_segment_packed('TYPE_UINT64', _ContextType, Tag, Segment0, Values) => 1227 protobuf_segment_convert(Segment0, packed(Tag, varint(Values))), !. 1228convert_segment_packed('TYPE_INT32', _ContextType, Tag, Segment0, Values) => 1229 freeze(Segment0, protobuf_segment_convert(Segment0, packed(Tag, varint(Values0)))), 1230 maplist(uint32_int32_when, Values0, Values). 1231convert_segment_packed('TYPE_FIXED64', _ContextType, Tag, Segment0, Values) => 1232 freeze(Segment0, protobuf_segment_convert(Segment0, packed(Tag, fixed64(Values0)))), 1233 maplist(int64_uint64_when, Values0, Values). 1234convert_segment_packed('TYPE_FIXED32', _ContextType, Tag, Segment0, Values) => 1235 freeze(Segment0, protobuf_segment_convert(Segment0, packed(Tag, fixed32(Values0)))), 1236 maplist(int32_uint32_when, Values0, Values). 1237convert_segment_packed('TYPE_BOOL', _ContextType, Tag, Segment0, Values) => 1238 freeze(Segment0, protobuf_segment_convert(Segment0, packed(Tag, varint(Values0)))), 1239 maplist(int_bool_when, Values0, Values). 1240% TYPE_STRING isn't allowed TODO: add it anyway? 1241% TYPE_GROUP isn't allowed 1242% TYPE_MESSAGE isn't allowed 1243% TYPE_BYTES isn't allowed TODO: add it anyway? 1244convert_segment_packed('TYPE_UINT32', _ContextType, Tag, Segment0, Values) => 1245 protobuf_segment_convert(Segment0, packed(Tag, varint(Values))), !. 1246convert_segment_packed('TYPE_ENUM', ContextType, Tag, Segment0, Values) => 1247 % uint64_int64_when(...), % TODO! https://github.com/SWI-Prolog/contrib-protobufs/issues/9 1248 freeze(Segment0, protobuf_segment_convert(Segment0, packed(Tag, varint(Values0)))), 1249 maplist(convert_enum(ContextType), Values, Values0). 1250convert_segment_packed('TYPE_SFIXED32', _ContextType, Tag, Segment0, Values) => 1251 protobuf_segment_convert(Segment0, packed(Tag, fixed32(Values))). 1252convert_segment_packed('TYPE_SFIXED64', _ContextType, Tag, Segment0, Values) => 1253 protobuf_segment_convert(Segment0, packed(Tag, fixed64(Values))). 1254convert_segment_packed('TYPE_SINT32', _ContextType, Tag, Segment0, Values) => 1255 freeze(Segment0, protobuf_segment_convert(Segment0, packed(Tag, varint(Values0)))), 1256 maplist(int64_zigzag_when, Values, Values0). 1257convert_segment_packed('TYPE_SINT64', _ContextType, Tag, Segment0, Values) => 1258 freeze(Segment0, protobuf_segment_convert(Segment0, packed(Tag, varint(Values0)))), 1259 maplist(int64_zigzag_when, Values, Values0). 1260% convert_segment_packed(Type, ContextType, Tag, Segment, Values) => % TODO: delete this clause 1261% domain_error(type(type=Type, % TODO: this is a bit funky 1262% context_type=ContextType), 1263% value(segment=Segment, 1264% tag=Tag, 1265% values=Values)). 1266 1267:- det(convert_segment/5).
Value from the combination of descriptor
"type" (in Type) and a Segment.
Reversible on Segment, Values.1272convert_segment('TYPE_DOUBLE', _ContextType, Tag, Segment0, Value) => 1273 Segment = fixed64(Tag,Int64), 1274 int64_float64_when(Int64, Value), 1275 protobuf_segment_convert(Segment0, Segment), !. 1276convert_segment('TYPE_FLOAT', _ContextType, Tag, Segment0, Value) => 1277 Segment = fixed32(Tag,Int32), 1278 int32_float32_when(Int32, Value), 1279 protobuf_segment_convert(Segment0, Segment), !. 1280convert_segment('TYPE_INT64', _ContextType, Tag, Segment0, Value) => 1281 Segment = varint(Tag,Value0), 1282 uint64_int64_when(Value0, Value), 1283 protobuf_segment_convert(Segment0, Segment), !. 1284convert_segment('TYPE_UINT64', _ContextType, Tag, Segment0, Value) => 1285 Segment = varint(Tag,Value), 1286 protobuf_segment_convert(Segment0, Segment), !. 1287convert_segment('TYPE_INT32', _ContextType, Tag, Segment0, Value) => 1288 Segment = varint(Tag,Value0), 1289 uint32_int32_when(Value0, Value), 1290 protobuf_segment_convert(Segment0, Segment), !. 1291convert_segment('TYPE_FIXED64', _ContextType, Tag, Segment0, Value) => 1292 Segment = fixed64(Tag,Value0), 1293 uint64_int64_when(Value, Value0), 1294 protobuf_segment_convert(Segment0, Segment), !. 1295convert_segment('TYPE_FIXED32', _ContextType, Tag, Segment0, Value) => 1296 Segment = fixed32(Tag,Value0), 1297 uint32_int32_when(Value, Value0), 1298 protobuf_segment_convert(Segment0, Segment), !. 1299convert_segment('TYPE_BOOL', _ContextType, Tag, Segment0, Value) => 1300 Segment = varint(Tag,Value0), 1301 int_bool_when(Value0, Value), 1302 protobuf_segment_convert(Segment0, Segment), !. 1303% convert_segment('TYPE_STRING', _ContextType, Tag, Segment0, Value) => 1304% Segment = string(Tag,ValueStr), 1305% protobuf_segment_convert(Segment0, Segment), !, 1306% ( false % TODO: control whether atom or string with an option 1307% -> atom_string(Value, ValueStr) 1308% ; Value = ValueStr 1309% ). 1310convert_segment('TYPE_STRING', _ContextType, Tag, Segment0, Value) => 1311 % TODO: option to control whether to use atom_string(Value,ValueStr) 1312 Segment = string(Tag,Value), 1313 protobuf_segment_convert(Segment0, Segment), !. 1314convert_segment('TYPE_GROUP', ContextType, Tag, Segment0, Value) => 1315 Segment = group(Tag,MsgSegments), 1316 % TODO: combine with TYPE_MESSAGE code: 1317 ( nonvar(Value) 1318 -> dict_pairs(Value, _, FieldValues), 1319 maplist(field_segment(ContextType), FieldValues, MsgSegments), 1320 protobuf_segment_convert(Segment0, Segment) 1321 ; protobuf_segment_convert(Segment0, Segment), 1322 maplist(segment_to_term(ContextType), MsgSegments, MsgFields), 1323 combine_fields(MsgFields, ContextType{}, Value) 1324 ), !. 1325convert_segment('TYPE_MESSAGE', ContextType, Tag, Segment0, Value) => 1326 Segment = message(Tag,MsgSegments), 1327 ( nonvar(Value) 1328 -> dict_pairs(Value, _, FieldValues), 1329 maplist(field_segment(ContextType), FieldValues, MsgSegments), 1330 protobuf_segment_convert(Segment0, Segment) 1331 ; protobuf_segment_convert(Segment0, Segment), 1332 maplist(segment_to_term(ContextType), MsgSegments, MsgFields), 1333 combine_fields(MsgFields, ContextType{}, Value) 1334 ), !. 1335convert_segment('TYPE_BYTES', _ContextType, Tag, Segment0, Value) => 1336 Segment = length_delimited(Tag,Value), 1337 protobuf_segment_convert(Segment0, Segment), !. 1338convert_segment('TYPE_UINT32', _ContextType, Tag, Segment0, Value) => 1339 Segment = varint(Tag,Value), 1340 protobuf_segment_convert(Segment0, Segment), !. 1341convert_segment('TYPE_ENUM', ContextType, Tag, Segment0, Value) => 1342 Segment = varint(Tag,Value0), 1343 convert_enum(ContextType, Value, Value0), % TODO: negative values: https://github.com/SWI-Prolog/contrib-protobufs/issues/9 1344 protobuf_segment_convert(Segment0, Segment), !. 1345convert_segment('TYPE_SFIXED32', _ContextType, Tag, Segment0, Value) => 1346 Segment = fixed32(Tag,Value), 1347 protobuf_segment_convert(Segment0, Segment), !. 1348convert_segment('TYPE_SFIXED64', _ContextType, Tag, Segment0, Value) => 1349 Segment = fixed64(Tag,Value), 1350 protobuf_segment_convert(Segment0, Segment), !. 1351convert_segment('TYPE_SINT32', _ContextType, Tag, Segment0, Value) => 1352 Segment = varint(Tag,Value0), 1353 int64_zigzag_when(Value, Value0), 1354 protobuf_segment_convert(Segment0, Segment), !. 1355convert_segment('TYPE_SINT64', _ContextType, Tag, Segment0, Value) => 1356 Segment = varint(Tag,Value0), 1357 int64_zigzag_when(Value, Value0), 1358 protobuf_segment_convert(Segment0, Segment), !. 1359 1360convert_enum(ContextType, Enum, Uint) :- 1361 uint64_int64_when(Uint, Int), 1362 proto_meta_enum_value_when(ContextType, Enum, Int). 1363 1364% TODO: use options to translate to/from false, true (see json_read/3) 1365int_bool(0, false). 1366int_bool(1, true). 1367 1368int_bool_when(Int, Bool) :- 1369 when((nonvar(Int) ; nonvar(Bool)), int_bool(Int, Bool)).
1372add_defaulted_fields(Value0, ContextType, Value) :-
1373 % Can use bagof or findall if we know that there aren't any
1374 % duplicated proto_meta_field_name/4 rules, although this isn't
1375 % strictly necessary (just avoids processing a field twice).
1376 ( setof(Name-DefaultValue, message_field_default(ContextType, Name, DefaultValue), DefaultValues)
1377 -> true
1378 ; DefaultValues = []
1379 ),
1380 foldl(add_empty_field_if_missing, DefaultValues, Value0, Value).1383message_field_default(ContextType, Name, DefaultValue) :- 1384 proto_meta_field_name(ContextType, _FieldNumber, Name, Fqn), 1385 proto_meta_field_default_value(Fqn, DefaultValue), 1386 % If the field is part of a "oneof" group, then there will be a 1387 % proto_meta_oneof entry for it (using the oneof_index). All 1388 % fields have a oneof_index, but our code doesn't depend on that. 1389 \+ (proto_meta_field_oneof_index(Fqn, OneofIndex), 1390 proto_meta_oneof(ContextType, OneofIndex, _)). 1391 1392add_empty_field_if_missing(FieldName-DefaultValue, Dict0, Dict) :- 1393 ( get_dict(FieldName, Dict0, _) 1394 -> Dict = Dict0 1395 ; put_dict(FieldName, Dict0, DefaultValue, Dict) 1396 ). 1397 1398:- det(combine_fields/3).
1408combine_fields([], MsgDict0, MsgDict) => 1409 is_dict(MsgDict0, ContextType), 1410 add_defaulted_fields(MsgDict0, ContextType, MsgDict). 1411combine_fields([field_and_value(Field,norepeat,Value)|Fields], MsgDict0, MsgDict) => 1412 put_dict(Field, MsgDict0, Value, MsgDict1), 1413 combine_fields(Fields, MsgDict1, MsgDict). 1414combine_fields([field_and_value(Field,repeat_packed,Values0)|Fields], MsgDict0, MsgDict) => 1415 ( get_dict(Field, MsgDict0, ExistingValues) 1416 -> append(ExistingValues, Values0, Values) 1417 ; Values = Values0 1418 ), 1419 put_dict(Field, MsgDict0, Values, MsgDict1), 1420 combine_fields(Fields, MsgDict1, MsgDict). 1421combine_fields([field_and_value(Field,repeat,Value)|Fields], MsgDict0, MsgDict) => 1422 combine_fields_repeat(Fields, Field, NewValues, RestFields), 1423 ( get_dict(Field, MsgDict0, ExistingValues) 1424 -> append(ExistingValues, [Value|NewValues], Values) 1425 ; Values = [Value|NewValues] 1426 ), 1427 put_dict(Field, MsgDict0, Values, MsgDict1), 1428 combine_fields(RestFields, MsgDict1, MsgDict). 1429 1430:- det(combine_fields_repeat/4).
Field - the assumption
is that all the items for a field will be together and if they're
not, they would be combined outside this predicate.
1441combine_fields_repeat([], _Field, Values, RestFields) => Values = [], RestFields = []. 1442combine_fields_repeat([Field-repeat-Value|Fields], Field, Values, RestFields) => 1443 Values = [Value|Values2], 1444 combine_fields_repeat(Fields, Field, Values2, RestFields). 1445combine_fields_repeat(Fields, _Field, Values, RestFields) => Values = [], RestFields = Fields. 1446 1447:- det(field_and_type/7).
ContextType and Tag to get the field name, type, etc.
1450field_and_type(ContextType, Tag, FieldName, FqnName, ContextType2, RepeatOptional, Type) =>
1451 assertion(ground(ContextType)), % TODO: remove
1452 assertion(ground(Tag)), % TODO: remove
1453 ( proto_meta_field_name(ContextType, Tag, FieldName, FqnName),
1454 proto_meta_field_type_name(FqnName, ContextType2),
1455 fqn_repeat_optional(FqnName, RepeatOptional),
1456 proto_meta_field_type(FqnName, Type)
1457 -> true % Remove choicepoint, if JITI didn't do the right thing.
1458 ; existence_error(ContextType, Tag)
1459 ).proto_meta_field_label(FqnName, _), proto_meta_field_option_packed(FqnName)
and set RepeatOptional to one of
norepeat, repeat, repeat_packed.1465fqn_repeat_optional(FqnName, RepeatOptional) => 1466 % TODO: existence_error if \+ proto_meta_field_label 1467 proto_meta_field_label(FqnName, LabelRepeatOptional), 1468 ( LabelRepeatOptional = 'LABEL_REPEATED', 1469 proto_meta_field_option_packed(FqnName) 1470 -> RepeatOptional = repeat_packed 1471 ; \+ proto_meta_field_option_packed(FqnName), % validity check 1472 fqn_repeat_optional_2(LabelRepeatOptional, RepeatOptional) 1473 ). 1474 1475:- det(fqn_repeat_optional_2/2).
proto_meta_enum_value('.google.protobuf.FieldDescriptorProto.Label', Label, _).1479fqn_repeat_optional_2('LABEL_OPTIONAL', norepeat). 1480fqn_repeat_optional_2('LABEL_REQUIRED', norepeat). 1481fqn_repeat_optional_2('LABEL_REPEATED', repeat).
proto_meta_enum_value('.google.protobuf.FieldDescriptorProto.Label', 'LABEL_REPEATED', _).
TODO: unused
1486field_descriptor_label_repeated('LABEL_REPEATED').proto_meta_enum_value('.google.protobuf.FieldDescriptorProto.Label', Label, _).1490field_descriptor_label_single('LABEL_OPTIONAL'). 1491field_descriptor_label_single('LABEL_REQUIRED'). 1492 1493:- det(term_to_segments/3).
Term, generating message segments1496term_to_segments(Term, MessageType, Segments) :- 1497 dict_pairs(Term, _, FieldValues), 1498 maplist(field_segment(MessageType), FieldValues, Segments). 1499 1500:- det(field_segment/3). 1501% MessageType is the FQN of the field type (e.g., '.test.Scalars1') 1502% FieldName-Value is from the dict_pairs of the term. 1503% TODO: Throw an error if proto_meta_field_name/4 fails (need to make 1504% sure of all the possible uses of field_segment/3 and that 1505% nothing depends on it being able to fail without an error). 1506field_segment(MessageType, FieldName-Value, Segment) :- 1507 ( proto_meta_field_name(MessageType, Tag, FieldName, FieldFqn), 1508 proto_meta_field_type(FieldFqn, FieldType), 1509 proto_meta_field_type_name(FieldFqn, FieldTypeName), 1510 proto_meta_field_label(FieldFqn, Label) 1511 -> true % Remove choicepoint, if JITI didn't do the right thing. 1512 ; existence_error(MessageType, FieldName-Value) 1513 ), 1514 ( proto_meta_field_option_packed(FieldFqn) 1515 -> Packed = packed 1516 ; Packed = not_packed 1517 ), 1518 field_segment_scalar_or_repeated(Label, Packed, FieldType, Tag, FieldTypeName, Value, Segment), 1519 !. % TODO: remove 1520 1521:- det(field_segment_scalar_or_repeated/7).
FieldType is from the .proto meta information ('TYPE_SINT32', etc.)1524field_segment_scalar_or_repeated('LABEL_OPTIONAL', not_packed, FieldType, Tag, FieldTypeName, Value, Segment) => 1525 convert_segment(FieldType, FieldTypeName, Tag, Segment, Value). 1526field_segment_scalar_or_repeated('LABEL_REQUIRED', not_packed, FieldType, Tag, FieldTypeName, Value, Segment) => % same as LABEL_OPTIONAL 1527 convert_segment(FieldType, FieldTypeName, Tag, Segment, Value). 1528field_segment_scalar_or_repeated('LABEL_REPEATED', packed, FieldType, Tag, FieldTypeName, Values, Segment) => 1529 Segment = packed(Tag,FieldValues), 1530 maplist(convert_segment_v_s(FieldType, FieldTypeName, Tag), Values, Segments0), 1531 packed_list(Segments0, FieldValues). 1532field_segment_scalar_or_repeated('LABEL_REPEATED', not_packed, FieldType, Tag, FieldTypeName, Values, Segment) => 1533 Segment = repeated(Segments), 1534 maplist(convert_segment_v_s(FieldType, FieldTypeName, Tag), Values, Segments). 1535% field_segment_scalar_or_repeated(Label, Packed, FieldType, Tag, FieldTypeName, Value, Segment) :- % TODO: delete this clause 1536% domain_error(type(field_type=FieldType, % TODO: this is a bit funky 1537% label=Label, 1538% packed=Packed), 1539% value(tag=Tag, field_type_name=FieldTypeName, value=Value, segment=Segment)). 1540 1541convert_segment_v_s(FieldType, FieldTypeName, Tag, Value, Segment) :- 1542 convert_segment(FieldType, FieldTypeName, Tag, Segment, Value). 1543 1544% Convert [varint(1,10),varint(1,20)] to varint(1,[10,20]). 1545packed_list([], []). 1546packed_list([T1|Ts], PackedList) :- 1547 detag(T1, Functor, Tag, _V1, List, PackedList), 1548 packed_list_([T1|Ts], Functor, Tag, List). 1549 1550% Functor and Tag are only for verifying that the terms are of the 1551% expected form. 1552packed_list_([], _, _, []). 1553packed_list_([T1|Ts], Functor, Tag, [X1|Xs]) :- 1554 detag(T1, Functor, Tag, X1, _, _), 1555 packed_list_(Ts, Functor, Tag, Xs).
MessageType's FieldName is defined as a map<...> in
the .proto file.
1560protobuf_field_is_map(MessageType0, FieldName) :-
1561 proto_meta_normalize(MessageType0, MessageType),
1562 proto_meta_field_name(MessageType, _, FieldName, FieldFqn),
1563 proto_meta_field_type(FieldFqn, 'TYPE_MESSAGE'),
1564 proto_meta_field_label(FieldFqn, 'LABEL_REPEATED'),
1565 proto_meta_field_type_name(FieldFqn, FieldTypeName),
1566 proto_meta_message_type_map_entry(FieldTypeName),
1567 assertion(proto_meta_field_name(FieldTypeName, 1, key, _)),
1568 assertion(proto_meta_field_name(FieldTypeName, 2, value, _)),
1569 !.DictTag{key:Key, value:Value} and a key-value list as described
in library(pairs). At least one of ProtobufTermList and Pairs
must be instantiated; DictTag can be uninstantiated. If
ProtobufTermList is from a term created by
protobuf_parse_from_codes/3, the ordering of the items is undefined;
you can order them by using keysort/2 (or by a predicate such as
dict_pairs/3, list_to_assoc/2, or list_to_rbtree/2.1580protobuf_map_pairs(ProtobufTermList, DictTag, Pairs) :- 1581 maplist(protobuf_dict_map_pairs(DictTag), ProtobufTermList, Pairs). 1582 1583protobuf_dict_map_pairs(DictTag, DictTag{key:Key,value:Value}, Key-Value)
Google's Protocol Buffers ("protobufs")
Protocol buffers are Google's language-neutral, platform-neutral, extensible mechanism for serializing structured data -- think XML, but smaller, faster, and simpler. You define how you want your data to be structured once. This takes the form of a template that describes the data structure. You use this template to encode and decode your data structure into wire-streams that may be sent-to or read-from your peers. The underlying wire stream is platform independent, lossless, and may be used to interwork with a variety of languages and systems regardless of word size or endianness. Techniques exist to safely extend your data structure without breaking deployed programs that are compiled against the "old" format.
The idea behind Google's Protocol Buffers is that you define your structured messages using a domain-specific language and tool set. Further documentation on this is at https://developers.google.com/protocol-buffers.
There are two ways you can use protobufs in Prolog:
.protofile: protobuf_parse_from_codes/3 and protobuf_serialize_to_codes/3.The protobuf_parse_from_codes/3 and protobuf_serialize_to_codes/3 interface translates between a "wire stream" and a Prolog term. This interface takes advantage of SWI-Prolog's dict. The
protocplugin (protoc-gen-swipl) generates a Prolog file of meta-information that captures the.protofile's definition in theprotobufsmodule, with the following facts:proto_meta_normalize(Unnormalized, Normalized)proto_meta_package(Package, FileName, Options)proto_meta_message_type(Fqn, Package, Name)proto_meta_message_type_map_entry(Fqn)proto_meta_field_name(Fqn, FieldNumber, FieldName, FqnName)proto_meta_field_json_name(FqnName, JsonName)proto_meta_field_label(FqnName, LabelRepeatOptional) % 'LABEL_OPTIONAL', 'LABEL_REQUIRED', 'LABEL_REPEATED'proto_meta_field_type(FqnName, Type) % 'TYPE_INT32', 'TYPE_MESSAGE', etcproto_meta_field_type_name(FqnName, TypeName)proto_meta_field_default_value(FqnName, DefaultValue)proto_meta_field_option_packed(FqnName)proto_meta_enum_type(FqnName, Fqn, Name)proto_meta_enum_value(FqnName, Name, Number)proto_meta_field_oneof_index(FqnName, Index)proto_meta_oneof(FqnName, Index, Name)The protobuf_message/2 interface allows you to define your message template as a list of predefined Prolog terms that correspond to production rules in the Definite Clause Grammar (DCG) that realizes the interpreter. Each production rule has an equivalent rule in the protobuf grammar. The process is not unlike specifiying the format of a regular expression. To encode a template to a wire-stream, you pass a grounded template,
X, and variable,Y, to protobuf_message/2. To decode a wire-stream,Y, you pass an ungrounded template,X, along with a grounded wire-stream,Y, to protobuf_message/2. The interpreter will unify the unbound variables in the template with values decoded from the wire-stream.For an overview and tutorial with examples, see
library(protobufs): Google's Protocol Buffers Examples of usage may also be found by inspecting test_protobufs.pl and the demo directory, or by looking at the "addressbook" example that is typically installed at /usr/lib/swi-prolog/doc/packages/examples/protobufs/interop/addressbook.pl