Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

P2464 Ruminations on networking and executors #1113

Closed
brycelelbach opened this issue Oct 5, 2021 · 2 comments
Closed

P2464 Ruminations on networking and executors #1113

brycelelbach opened this issue Oct 5, 2021 · 2 comments
Labels
B1 - focus Bucket 1 as described by P0592: material that is mentioned in this plan. executors IS Ship vehicle: IS LEWG Library Evolution networking Networking ready-for-library-evolution-meeting-review This paper needs to be discussed at a Library Evolution meeting SG1 Concurrency size - small paper size estimate

Comments

@brycelelbach
Copy link

P2464R0 Ruminations on networking and executors (Ville Voutilainen)

@brycelelbach brycelelbach added LEWG Library Evolution SG1 Concurrency networking Networking executors IS Ship vehicle: IS B1 - focus Bucket 1 as described by P0592: material that is mentioned in this plan. ready-for-library-evolution-meeting-review This paper needs to be discussed at a Library Evolution meeting size - small paper size estimate labels Oct 5, 2021
@brycelelbach
Copy link
Author

brycelelbach commented Oct 5, 2021

2021-10-04 Joint Library Evolution and Concurrency Telecon

P2464R0: Ruminations on networking and executors, and responses (P2469R0)

P2464R0: Ruminations on networking and executors

P2469R0: Response to P2464: The Networking TS is baked, P2300 Sender/Receiver is not

P2300R2: std::execution

P2444R0: The Asio asynchronous model

2021-10-04 Joint Library Evolution and Concurrency Telecon Minutes

Chair: Bryce Adelstein Lelbach

Champion: Ville Voutilainen (for P2464) and Christopher Kohlhoff (for P2469)

Minute Taker: Gašper Ažman

Start P2464 Presentation: 2021-10-04 15:04 Pacific

End P2464 Presentation: 15:36

Start P2469 Presentation: 15:38

End P2469 Presentation: 15:49

End P2464/P2469 Discussion: 16:00

Networking and Executors Polls

To be taken electronically:

POLL 1: The Networking TS/Asio async model (P2444) is a good basis for most asynchronous use cases, including networking, parallelism, and GPUs.

POLL 2: The sender/receiver model (P2300) is a good basis for most asynchronous use cases, including networking, parallelism, and GPUs.

POLL 3: Stop pursuing the Networking TS/Asio design as the C++ Standard Library's answer for networking.

POLL 4: Networking in the C++ Standard Library should be based on the sender/receiver model (P2300).

POLL 5: It is acceptable to ship socket-based networking in the C++ Standard Library that does not support secure sockets (TLS/DTLS).

Summary

We finished this 3-week review of Networking and Executors with presentations of three papers:

  • P2300R2, an updated revision of the sender/receiver model addressing feedback from our summer review.
  • P2464R0, a position paper suggesting that there are deficiencies in the Networking TS/Asio async model and that it should not be adopted by the Standard Library.
  • P2469R0, a rebuttal of P2464 that argues that the Networking TS is proven, mature, and ready to ship, while the P2300 sender/receiver model is not, and suggesting that the sender/receiver model could be layered on top of the Networking TS/Asio model.

One major topic of today's discussion was layering - can sender/receiver be built on top of the Networking TS/Asio model, and vice versa. One of the principle questions with such a layering is how error channels and cancellation would be supported.

Error handling was a main point of debate between proponents of the two models. Some claimed that the Networking TS/Asio model did not provide sufficient mechanisms for handling errors during work submission.

Performance overheads and efficiency were another point of contention. Proponents of the Networking TS/Asio model feel that implementing and using async APIs with the sender/receiver model would come with unacceptable overheads associated with the construction of sender/receiver objects.

There were some questions about the change in P2300R2 from having a maybe-eager and always-lazy version of each algorithm to having only the strictly-lazy version. This warrants further discussion when we next look at the sender/receiver model in detail.

Outcome

We will take electronic polls on Networking and Executors to determine how to proceed. Details on how to vote have been sent to the lib-ext@ mailing list.

@brycelelbach
Copy link
Author

brycelelbach commented Nov 3, 2021

2021-10 Library Evolution / Concurrency Polls on Networking and Executors

Consensus and outcomes determined by @ben-craig and @FabioFracassi.

Poll 1: The Networking TS/Asio async model (P2444) is a good basis for most asynchronous use cases, including networking, parallelism, and GPUs.

Strongly Favor Weakly Favor Neutral Weakly Against Strongly Against
5 10 6 14 18

Outcome: Weak consensus against.

What this doesn't mean: It doesn't mean that the NetTS async model isn't a good fit for networking. There were many comments to the contrary.

What this means: If the authors continue to pursue a design similar to the current NetTS, they would have an easier time getting consensus by focusing on the networking side of things, rather than proposing the design for general asynchrony.

Poll 2: The sender/receiver model (P2300) is a good basis for most asynchronous use cases, including networking, parallelism, and GPUs.

Strongly Favor Weakly Favor Neutral Weakly Against Strongly Against
24 16 3 6 3

Outcome: Consensus in favor.

What this doesn't mean: It doesn't mean that S&R is going into C++23 (though it might). It doesn't mean that P2300 as it stands today will be sent forward to LWG. It doesn't prevent us from adding new asynchronous models in the future.

What this means: Work will continue in LEWG on refining P2300, and LEWG will keep the various asynchronous use cases in mind while working on P2300.

Poll 3: Stop pursuing the Networking TS/Asio design as the C++ Standard Library's answer for networking.

Strongly Favor Weakly Favor Neutral Weakly Against Strongly Against
13 13 8 6 10

Outcome: No consensus

What this doesn't mean: The NetTS is not "dead".

What this means: WG21 will still work on networking in this general form, but the authors need to do a lot in order to build up consensus to get something like the TS merged into the standard. The bulk of this work should be done in SG4. Many of the people in favor of stopping work on the TS would like networking to be built on top of Senders and Receivers. Others were opposed to the lack of security through Transport Layer Security (TLS). It is highly unlikely that design changes to the networking TS can be made fast enough, and consensus gained fast enough, for networking to make C++23.

Poll 4: Networking in the C++ Standard Library should be based on the sender/receiver model (P2300).

Strongly Favor Weakly Favor Neutral Weakly Against Strongly Against
17 11 10 4 6

Outcome: Weak consensus in favor.

What this doesn't mean: Work on networking using other models will still be reviewed and considered on its own merits. WG21 doesn't pause work on a concrete paper based off of the wish for another paper. Note that there are a high number of neutrals on this vote. Many of the neutrals (and some of the abstentions) would like to see a paper before picking a side here.

What this means: In the short term, this poll result doesn't mean much. We don't have a paper in hand that proposes networking based on the P2300 model. For paper authors though, this poll is encouragement to do work in the area of networking based on senders and receivers, or to be prepared with compelling new information on why networking should use a different model.

Poll 5: It is acceptable to ship socket-based networking in the C++ Standard Library that does not support secure sockets (TLS/DTLS).

Strongly Favor Weakly Favor Neutral Weakly Against Strongly Against
9 13 5 6 13

Outcome: No consensus.

What this means: A networking library that does not support secure sockets will face significant headwinds getting through the standardization process.

What this doesn't mean: This doesn't make a statement on whether insecure networking should be included, the defaults of secure vs. insecure, or how things like ABI should managed.

Guidance for the Networking Study Group

Before bringing networking papers back to LEWG, two major areas need to be thoroughly addressed: Security, and the Senders and Receivers async model.

Those voting in favor of Poll 5 argued that the insecure parts are the building blocks for the secure parts, and that ABI is major concern that will plague TLS support. Those voting against Poll 5 argued that secure sockets are needed for communicating with many sites on the internet, and that shipping without secure sockets would be irresponsible. A networking proposal needs to address these concerns before coming to LEWG.

As for networking in combination with Senders and Receivers, I will show this older poll from the 2021-09-28 telecon:

POLL: We believe we need one grand unified model for asynchronous execution in the C++ Standard Library, that covers structured concurrency, event based programming, active patterns, etc.

Strongly Favor Weakly Favor Neutral Weakly Against Strongly Against
4 9 5 5 1

Outcome: No consensus (leaning in favor).

The combination of this "grand unified model" poll and Poll 4 heavily encourages the networking study group to produce a paper based on Senders and Receivers. There is room to produce a non-S&R paper, but such a paper would need to provide compelling new information in order to convince the "grand unified model" contingent that S&R can't get the job done suitably.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
B1 - focus Bucket 1 as described by P0592: material that is mentioned in this plan. executors IS Ship vehicle: IS LEWG Library Evolution networking Networking ready-for-library-evolution-meeting-review This paper needs to be discussed at a Library Evolution meeting SG1 Concurrency size - small paper size estimate
Projects
None yet
Development

No branches or pull requests

1 participant