Bidders Conference Transcript 6-03-09 Notes: 1. Bidders Conference was held June 3, 2009 at USC/ISI 2. Presenters included members of the RFC Editor staff - Bob Braden, Sandy Ginoza and Alice Hagens; and Ray Pelletier, IETF Administrative Director 3. Transcript should be read in conjunction with the presentation material located under Bidders Conference at: http://iaoc.ietf.org/rfced-procurement.html 4. Minor editing was done to make more readable. No questions or answers were modified. 5. As noted herein the final, official responses to questions are posted under Bidders Conference Questions & Answers at the address above. The responses in the transcript therefore may -not- be the final, official answers. 6. The last date for submitting questions has been changed from 5 June to 9 June to rpelletier@isoc.org Ray Pelletier IETF Administrative Director June 5, 2009 /Transcription Begins >> June 3, 2009, 10 a.m. IETF RFC bidders conference. >> Welcome to ISI. I'm Bob Braden. Why don't we go around the room and identify ourselves. SANDY GINOZA: I'm Sandy Ginoza, with RFC Editor. ALICE: Alice Hagen, also with RFC Editor. >> Stacy Burns with RFC Editor. >> Paul Hoffman with Standcore. >> Bob Hinden I'm the IAOC chair. >> Karen Morlan with AMS >> Kirsten Machi with AMS. >> Mary Moore with Wipro >> Renee, the transcriptionist. >> Ray Pelletier, I'm the IETF administrative director, work with the I A O C. And is this Susan on the phone? NEW SPEAKER: Yes. >> RAY PELLETIER: Could you identify yourself. NEW SPEAKER: Sorry. Susan in Boston. RAY: Could you spell that, please. >> H U N Z I K V R. RAY PELLETIER: Thank you very much. John, are you there? Okay. SANDY GINOZA: I think John was going to join later because they had an IAB phone call. Is Meghan on line? SANDY GINOZA: Meghan is one of our editors, she's located in Boston. RAY PELLETIER: Terrific. Well, you're disbursed. In fact Alice is disbursed, she's on the East coast. BOB: Yes. Not right now. RAY PELLETIER: Except right now, this is a hologram, actually. Thanks, Bob. BOB BRADEN: It is in fact perhaps of some interest that, having editors dispersed, works, if they are well trained. And are the organization and understand, and are the people that understand the rules, and in that case, it has worked very well. So, I guess I should go to the next. Okay. So, this is a list of the staff. Who number currently 4.75 FTEs. I am the leader of the group. Project leader here at ISI I work quarter of my time spent on RFC editor. Sandy and Alice and Meghan are full time. Stacy is full time. Craig Ward is a programmer that works 30 percent for us. And Alba is our staff support. Let's see, we did this. BOB: So we're going to do this in a different order. RAY PELLETIER: We're all set, yes. RAY PELLETIER: That was the welcome and introductions part. This is the process and agenda part. And let's see here. BOB BRADEN: It's not a bad idea to get ahead of schedule, because -- RAY PELLETIER: Okay. One of the things to add to the agenda which was on the first slide there is that there will be a wrap up at the end. Remind me. Because I think we're finishing up with what the Q and A part but I want to make sure that we do get to the wrap up. And, NEW SPEAKER: This is Susan, I can't seem to get, it looks like the server is down. Am I the only person line who cannot get through to the web site? I tried through internet, and neither one of them is finding it. I'm having a hard time following. RAY PELLETIER: Okay. So go to I A O C Dot IETF.Org. NEW SPEAKER: Yes, that's where it stops. RAY PELLETIER: Well, don't do a WWW. NEW SPEAKER: Right. And, and I didn't do. I did it with and without it. And it doesn't want to go there. RAY PELLETIER: And, I know I am online there. -- NEW SPEAKER: Hold it. Now it came up. Never mind. Keep going. RAY PELLETIER: Okay. The purpose of the conference is to provide people an opportunity to address the current RFC Editor in person, or the phone, and ask questions. In preparation for the, you know, submission of a proposal. Some rules here I want to make sure we get covered, a reasonable attempt will be made to answer the questions during the conference. However it may not be possible. And, consequently, if that's the case, we are transcribing the session, we will review the questions, we'll provide answers, the answers will be published by the 12 June, that's our deadline, 12 June. And, the other thing is, that the, while we'll provide oral answers today. The published answers are the official responses. Okay. So we reserve the right to correct ourselves. To extend or modify our remarks as they do in Congress, for example. >> Today is not the last day to ask questions. Frankly, I'd be surprised if you didn't learn something from today's materials, and still have some questions as a result of having more information. The deadline for submitting questions, however, is Friday. June 5th. Okay. And, as long as they're dated June 5th, I don't care what time they come in. Because then we'll take the next week in order to come up with the answers and publish them. PAUL HOFFMAN: Ray, quick question, if there aren't many questions, and you're ready, will you publish them beforehand are or are you going to wait until Friday, do you think. Start of the clock. RAY PELLETIER: Since the process by which we would be reviewing the questions and answers, is going to involve several different groups of people, it probably will be the 12th. I'll take probably every minute of it. As I only promised midnight the 12th, someplace in the world. PAUL HOFFMAN: Right. NEW SPEAKER: Unless of course we get no questions. RAY PELLETIER: They'll be published at the IAOC web site, dash procurement link, where we put all of the information, concerning this whole process to provide some transparency to it. Okay. So there's going to be a bunch of presentations, the Q and A period will follow the presentations. Please identify yourself, we do have the transcriptionist, Renee here. If we don't think we answer your question, please say so. And we'll try to reformulate our response. And if we still don't answer your question, we'll try by the 12th to do as good a job as we can. Again, those are the official responses, the one on the 12th. I have received some questions in advance, and I indicated to folks, if they send them to me in advance of the conference, then, the questions, the questioner, would remain anonymous, so that we don't know who is asking the question. But, obviously, if you're asking a question now, I'm know who you are. And so will everybody else. BOB BRADEN: Which hopefully will not inhibit you from asking questions. RAY PELLETIER: No, it shouldn't. Was that the last one? Let's see. 5 of 5. How's that. What's next on the agenda. Restructuring or the history. SANDY GINOZA: History first, that's how we were going to do it. RAY PELLETIER: Yes, thanks, that makes sense before the restructuring stuff. It's obvious we rehearsed this several times before we got here. PAUL HOFFMAN: Very much like IETF meeting. RAY PELLETIER: We want to make sure you get the flavor of the IETF. PAUL HOFFMAN: Including not being able to get the projector working. BOB BRADEN: So, whoever wins the bid is going to take over a process which has resided here on this floor of this building for many years. And it's the system we're going to talk about today, is an evolution of 28 years or so of experience. It's continuously evolved. We, I would expect that, that whatever you do will probably differ in the next few years from what we could now. And I know some of you have other ideas on how things ought to be done. We have taken a conservative attitude in general, we don't change things unless there's a strong reason to do so. And I would urge you you to do the same thing. [Slide 9] Anyway, the RFC documents series, as you probably know was begun by Steve Crocker [RFC 1] and Jon Postel in 1969, the RFC were the first document series to be published online, because the people who published them were the people who invented online. They started out as informal memos, if you probe back to the first few RFCs, some of them are just meeting announcements and so on. Jon was an archivist by nature and he kept a notebook. And he started writing down numbers and keeping track of RFCs and he did it for 28 years until he very sadly died in Santa Monica in Saint John's Hospital in 1998. But he was one who established and maintained a style and quality. And, it's fascinating fact that he was in fact a two finger typist. So that's Jon the left and Steve in the middle and Vint Cerf on the right. And picture of Jon. He was was an iconic figure around the IETF, and is sorely missed. [slide 11] So, the RFC series began No. 1, as the ARPANET was beginning, in 1969; around RFC 700 the TCP/IP research began and initially it was a separate document series. IENs were used initially for TCP. One of the lessons we learned was having two different document series was a really bad idea. Because it was confusing knowing which series you wanted. So the I E N series is terminated although they're still on line on the web site. And everything folded into one series. And I think that's a lesson to remember. The Internet itself was born, became real, one January 1983, and we're up to RFC 830. The IETF was created in 1985 and we're up to RFC 950. And then the Kobe incident, which resulted in modern IESG and IAB around RFC 1400. And when Jon passed away we were at RFC 2430. In '93, it was 1400. And today we're up to 5400. The average document is 30 pages. That's a lot of documents, a lot of pages of stuff. [slide 12] This is a graph that shows the Ray that the arpanet, and the period and in internet period. This one year, was it 2005, was anomalous, I'm not quite sure why. SANDY GINOZA: 2006. BOB BRADEN: Well, the IETF was just very productive, or they wrote a lot of words that year. [slide 13] I want to say a little bit about April 1st RFCs, that this is one of the RFC editors private activities. It is not sanctiond by anybody. We just do it. They're not published as internet drafts first, they're not approved by anybody. The RFC editor and plus, we tap some consultants on this floor who have senses of humor, reviews the submissions. We usually get, 5, 8 submissions every year. And we select from, between zero and 3-4 usually to publish. If they're unsuitable, we don't publish them. Most RFCs that are dated April 1st are satirical documents. However, there are some which are actual real, real specs. And we assume that the IETF members are able to distinguish. We have a certain style, we like them serious. I mean, no joking ones. People often submit really joke ee ones, sort of juvenile. And that's not on. They have to be serious. They have to, when people look at it, you have to first think, maybe this is serious, or this is serious, and then, kind of gradually realize, no, this is not serious. So avian carriers is our most famous, and evil bit is my favorite. PAUL HOFFMAN: You shouldn't assume that people even at the end of reading them that people do not believe are serious. You did do one last year that people took serious, Jon and Harold's internationalization one. We still get people who think it's serious. BOB BRADEN: Perhaps we went too far. PAUL HOFFMAN: Or they didn't go far enough. Most of us were horrified on the first paragraph. But other people finished it and said, wow, that's really interesting. BOB BRADEN: It often happens we get a submission that we like but is not quite right. And we try to talk to the author to get it where we want it. So the arpanet and we had research effort to production to commercial effort and the technical community grew and the IAB created the IETF in 19 58. And then the IETF ate its parent and started over in 1982 and through all this up upheaval which preceded your time by a lot. The RFC editor kept on publishing and adapting to the changing environment, but trying to maintain the consistency, quality and integrity of the series. Which ever one, the one who inherits the mantle, we hope will continue that. [slide 15] So, we, the RFC Editors at ISI, we have had a dual loyalty, quite frankly, to the IETF process, which we serve, and to the RFC series, because it's partly, it's an emotional reaction in the legacy of Jon's and we revered very much and because we think it's a great endeavor. It's neat we think. RAY PELLETIER: Or it could be the Stockholm syndrome? Where you you became, the prisoners became enamored with their captures. BOB BRADEN: Well, maybe. So, because of its history, Jon in particular had a lot of autonomy, he basically did what he wanted. He took some input but basically did what he wanted. If the RFC editors wanted to change headers or change the web site, we could just do it. We, in fact were very conservative. We didn't do things that would upset people. We listened carefully to what people said. But, we had a considerable autonomy. A lot of that has been reduced by all the RFCs procedural RFCs that have been written recently and the new restructuring as Ray will talk about has further reduce it and it's probably unclear to me what the future is going to be in that area. There's, we create, we created an editorial board, because as another avenue to get input from the community, but in this case, aboard of wise men and women, old farts, who had a lot of experience with the internet, with publishing, and is not an official IETF body, although there has been some discussion in making it one. And it's particularly used, I particularly use it for independent submission reviews. But, they, they have, it has somewhat, quite a bit of autonomy from us. [slide 17] So after Jon died, Joyce Reynolds and I took over the function, became co-RFC editors. Co-incidently, so the RFC editor had been funded by the government until up to this point, by DARPA actually. As a holdover from the internet research effort. But 1998, DARPA decided that it was internet was no longer an experiment. And they would not fund the RFC editor anymore. Fortunately, the Internet Society stepped up to the plate and began funding us. So the current leadership is me, her and her (Sandy & Alice). And, 40 years old. It's an archive series, RFCs are forever. And there's a trove of technical history. Sometimes documents don't get published - often for bad reasons. But anyway. PAUL HOFFMAN: Say more about that. Say more about what you just sort of mumbled there, because I think that is valid. BOB BRADEN: I'll talk to you about off line. There are cases where things which should have been recorded as RFCs, because of the historical value, were not because the, the author for one reason or another refused. PAUL HOFFMAN: Oh. BOB BRADEN: Even though we tried very hard to convince them. Some very stubborn people on the internet. PAUL HOFFMAN: You've noticed that by yourself, have you. BOB BRADEN: I won't say anymore about that. [slide 19] So, we have a web site. It has been a source of disappointment to me personally. We put a lot of effort into the web site. It's a little creaky now, but it is not up to modern elegant standards of web re, but, it's functional. I think. It's been a source of disappointment to me that the IETF is basically ignored it and used their own web site. So we have the primary, the RFC editor, historically has had the primary repository of documents. And the IETF repository is derived from ours. And conversely, they have the repository of internet drafts and we mirror them on our web site. We have the master index, we keep the, what's called the official internet protocol standards list, which is somewhat of an historical item. But it's a list of all the protocols, in which the, those which have been obsolete ted have been struck. [slide 20] This is a record of the submitted and published rate. Ray loves this picture. RAY PELLETIER: I do. SANDY GINOZA: If I can interject quickly. It's just the more recent, so last 5 or on six years. And one of the questions I saw on the list that you sent us yesterday was about, type of workload and why there are bursts. And so, I think the better responder for that would probably be the IESG or Russ maybe. But what we can see from here, and I think that Russ or IESG said this before, is that a lot of ADs when there's going to be turn over, try to clear their plate for the incoming AD. So typically March we see a spike. If you you look at the red lines -- NEW SPEAKER: I'm having a hard time hearing that, could you be closer to the microphone. Is SANDY GINOZA: Can you hear me. NEW SPEAKER: Yes. SANDY GINOZA: So what we're seeing in this diagram, is submissions, and typically, I think the ADs try to clear out what's on their plate before the next AD comes into office or position. So, a lot of times there's a spike in March, for approvals, so we'll receive a lot of document submissions in March. And a couple, a month or so later, we'll see a spike in publications. BOB BRADEN: Yes. So one of the problems you face in managing this effort is that it is bursty. And, you have to watch the queues carefully, because we got in trouble a couple of years ago. We like to accuse Bill, without taking any action, and we definitely got in trouble and people complained quite rightly. We worked really hard to get out of that, and but, you have, but it is difficult. It's also difficult from a staffing point of view. Because it means variable load on the staff. To be blunt about it, I don't think that Ray has been as sympathetic to that problem. RAY PELLETIER: Strike that. SANDY GINOZA: One other thing I wanted to tell you guys. We actually took an average for per month publication submissions, and it was averaging about 28 submissions and publications. I think the last couple of years, the IESG has sort of made an even distribution of submissions, there haven't been serious spikes in submission, but again, I don't know how that's controlled. PAUL HOFFMAN: Do you see any spikes on independent submissions or is that fairly even? BOB BRADEN: That's even. It's random. It's random process. PAUL HOFFMAN: Yes. [slide 21] BOB BRADEN. So I thought I should just say something quickly about sub series, because this is a bit of history which we're carrying around with us. RFC of course are labeled numerically. But, at one point Jon felt the need for labeling subsets, important subsets of the RFCs. And so he called them sub series, of which there are three. BCPs, STD which are the standards, full standards documents. And the FYIs which were user documentation. And actually, that was the original purpose of this. So, originally, we, the IAB believed that all protocol specs would quickly reach full standards status. And then the, so the STD sub series would include all significant standards documents. Of course, that didn't happen. Most standards never get beyond proposed standards. So, STD are not all that useful. RAY PELLETIER: That doesn't seem to make much difference to the marketplace though, it stayed a proposed standards. They used it. BOB BRADEN: That's quite true. In fact the marketplace is quite happy to implement internet drafts. [slide 23] So, furthermore, Jon overloaded the STD numbers to represent standards, complete standards. So, that one STD number can contain multiple RFCs. So STD 5 is IP and it includes 791, 9, all those different RFCs. And, if you look up a, an STD 5, you will find a file which contains a concatenation of the text of all these documents, with a note at the beginning that says, by the way this contains more than one RFC. And so, this also possible that there are some STD numbers that are empty. STD 12 was originally network time protocol, but the network time protocol document, it was obsoleted and was not replaced but the thing which obsoleted it is not a full standards, and therefore it cannot be STD so the STD No. 12 is reserved, and there are a bunch of reserved STD numbers. BCPs, which are almost standards, are treated the same as STDs, there can be muliple RFCs per BCP. That's a later feature. [slide 24] So, another comment is that RFC numbers is really the name of a document. But people often call it as, treat it as if it were the name of a protocol. But RFC numbers are not really useful name for protocols, because you have multiple RFCs relating in complex ways to make a protocol. And a proposal that IETF had a few years ago talked about naming standards ISDs, personally, I think that was a great idea and I wish the IETF had not ground to a halt on that. But, the, ... it, it is an area in which I hope things will get clarified and, I mean, I don't think any other standards body is so muddled about this problem. They all have found different ways to name standards, and have multiple documents per standard. So, I hope the IETF will solve that problem and however they do it will affect you. Okay. SANDY GINOZA: We're going back in time a little bit so Ray can do the restructuring. RAY PELLETIER: Yes. If I can find my way through Adobe. BOB BRADEN: Ray is about to describe a large experiment. We all hope it succeeds. RAY PELLETIER: This is an overview and hopefully, you'll have read all the other documents that back stop this particular discussion. Which is, this one right here, for example, this draft IAB- RFC-editor-model-07, which I believe is out on the street now, and still has some changes and little nits and stuff that are being done to it. NEW SPEAKER: Are you talking about something that's in the slides? RAY PELLETIER: Yes. NEW SPEAKER: Okay. Sorry, I couldn't quite hear where you're going. RAY PELLETIER: I'm on the, the Adobe file on RFC editor restructuring. NEW SPEAKER: Right. I'm there. Okay. RAY PELLETIER: So it says goverened by. I should have numbered my slides. Thank you, for calling my attention to that. NEW SPEAKER: That's okay. It's hard when you're not in the room. RAY PELLETIER: Yes. So, this is right now, this is the guidance for the document. As a matter of sequence it's always nice to have an RFC number that you're building to, as opposed to a draft which continues to get updated. But in any event, here is where we're at. My expectation is perhaps before June 29 when the proposals are due there will be an RFC number attached to this. Which means these guys have to work really hard to get the editing done. SANDY GINOZA: My understanding is that it's being held up by the 3932bis document currently. RAY PELLETIER: I hope not. But you're probably right. So the drafter of this is the internet architecture board. Not the IETF architecture board. It's the IAB. They are the organization that is responsible for the RFC Editor in the sense that the RFC editor is not an IETF organization. The RFC editor exists to publish RFCs regardless of what the source is. It 's just that the main body of work just happens to flow through the IETF. So, the IAB, as the first line says, on behalf of the internet technical community, and not on behalf of the IETF, what were they trying to do? They want to increase flexibility and operational support options, provide for the orderly succession of the RFC Editor, and insure the continuity of the series. Maintain the quality, very important. Maintain the timely processing. And things turnaround very quickly now, in fact, at some point in time, I think within the last six months, they said, could you slow it down. And I thought, you know, that's a big difference from 2006 and 2005 and now. Insure document accessibility. RFCs are for everybody, they're free, they go up there and you know, obviously the goal is to make sure that people have access to them. Find a way to reduce cost and increase cost transparency. So, the four functions; currently, ISI is all of this. Okay. All of this. They're doing it all. So Bob Sandy and Alice and the rest of the team are doing every part of this. The IAB through a discussion with the broad community and certainly the IETF, has decided that they feel like those objectives met with this kind of a structure, creating four different entities. The RFC Series Editor, with an RFC Series Advisory Group. The Independent Submissions Editor, with an optional independent submissions editorial board. There is an editorial board right now as Bob described. And this one envisions that there's perhaps two different groups. The RFC Production Center, and the RFC Publisher. There's four RFC streams, these streams exist now, this isn't new. The IETF stream, which is managed by the IETF Engineering Steering Group, the independent submission stream, which is has been managed by Bob and his team, which will now be managed by the independent submissions editor. The IRTF, and that's the research task force, is managed by their steering group and the IAB is managed by that group itself. The IAB. All four of these streams under the new structure, all four of these streams are going to go through the production center. So there's no change there, they're all going to come through the same production center as they have been coming through. And then from the production center, they go to the publisher. And the publisher is responsible for a lot of different things, but announcing the publication of the RFC and also maintaining them and archiving them et cetera. More details to follow. The RFC series editor - this information is in the model document. But RFC series editor is responsible for identifying the appropriate steps for the continuity of a series. Exercise executive level management over the implementations of the policies, processes and procedures, to insure the quality and consistency of the series and works with the RSAG, the advisory group, and where appropriate the IAB and IAOC to develop new policy and to see that contractual greets are met. The IAOC is responsible to the IETF for considerable matters. Entering into contracts with different bodies. And, I'm the IETF administrative director and it's my responsibility to do RFPs and manage contracts and negotiate contracts, and that's where the IAD fits in the world. >> The RFC series editor will take proposed changes to the community for discussion. They will participate in the reviews of the publisher, the production center and the independent submission editor to ensure continuity. So there's a management role here of the whole series and the processes and procedures. Develops, maintains and publishes the RFC style manual for use by authors, editors, the stream managers, the production center and the publisher. So there's one guide book that all folks are to develop their RFCs, their internet drafts in accordance with. And the production center is to edit according to that, for example. The RSE manages the RFC errata process. It happens from time to time, that RFC will be published and people will note that there are editorial problems or technical issues with the document. And they'll report those and those are reported online and the appropriate stream manager will review those errata to determine whether it's acceptable or not acceptable or should be held for a future change to the RFC for example. I think there's going to be some discussion about that in a little bit. The RSE oversees the consistency of the consistency of the RFCs within a series, so looking at a historical style over 5400 RFCs and the RFC style manual. Here's the schematic of the description I just gave. See at the top, we have the stream producers, the IETF, the IAB, the IRTF and the community at large. Anyone out in the community that want to write an RFC has the ability to do so. It doesn't mean it's going to get there, but they have the ability to approve it. The stream approvers are the IESG, IAB, IRSG, and the ISE who is assisted by the independent stream editorial board. All of those streams feed into the RFC production center. All the editing work takes place and the formatting and consistency for style with the RFC style manual. And then goes to the RFC publisher for publication online, and archiving and indexes, et cetera, et cetera. The RFC series editor, with the advisory group, with his or her advisory group, overseas this process. And works with these folks to improve the process. And now, PAUL HOFFMAN: Ray, question. So, Bob mentioned earlier about the RFC editor.Org web site. And that it is a place that still many people come to first, which I still think is good. Who is responsible for that? Do you believe, in that? Is it the RFC editor the production center? I mean for design and adding new things and such like that. RAY PELLETIER: Two things about that. One is, it is the official RFC web site. That is the official web site. It's not tools Dot IETF.Org. It is the RFC editor.Org web site. It's intended that that web site would be maintained by the RFC publisher. PAUL HOFFMAN: Maintained. I'm talking about designs and features added and such. RAY PELLETIER: The evolution of that will involve the RFC series editor and other participants in the community. In a way similar perhaps to the way that the IETF.Org web site is currently being modified. And, if you haven't seen the evidences of that, I think sometime in the next month or so, you'll see some changes. I'm not sure this is going to change like that, but -- Bob? BOB BRADEN: So, whose budget will support the system programmer who does the work? PAUL HOFFMAN: Thank you. RAY PELLETIER: So, the RFC editor is now currently funded or, under contract with I A S A through ISOC. The ISOC assigns all of the contracts. And all of the funding for all of these bits will come through the same process. BOB BRADEN: In the future. RAY PELLETIER: Same thing. PAUL HOFFMAN: I think Bob was asking the question that I was trying to ask, which is you now have four functions up there who will have individual bids. One of them will be paying somebody to design the site, add new tools and such, and therefore, is going to need to know, which, you know, how much to bid or not to. If somebody is bidding on just one of them and it's not theirs then they know not to. RAY PELLETIER: So the changes, that, the development work will be done by the publisher under contract with IASA. RAY PELLETIER: And I think we're now ready for independent submissions, if I answered the question. SANDY GINOZA: Actually we're moving into how an ID becomes an RFC. RAY PELLETIER: Right. The meat of the whole thing. Thanks, Sandy. SANDY GINOZA: Just so you guys know, we're going to try to breeze through some of these, some of these slides, because we have a demonstration later. So, it will make a lot more sense in context. If you guys have questions, you can stop us but it also might help if you mark the slide number and maybe when we're going through the demonstration, it might answer your question are on you can stop us as we sgo through the demonstration. So, how an ID becomes an RFC. There's four possible streams as has already been mentioned a few times. We get them from the IAB, the IETF, which is the majority of the work that we do. Independent submission stream and the IRTF. [slide 27] This is flow chart for a lot of the work that we do. A lot of the different states, there actually should be four green boxes, the independent submissions and the IRTF is missing from this diagram. There's a lot of circles or cyclic cycles here going on. Things can go back, we can ask questions to the authors, we can send them back at any time. BOB BRADEN: We should mention that this is a very much over simplified view of what actually happens sometimes. A lot of exceptions. SANDY GINOZA: Yes, so, this is a simplified process, and this picture is even more simplified process. [slide 28] So, what we we did is, there's so many exceptions, the majority of documents are exceptions, we took simple case, typical case straight through the processes, we noted submission process for example, the IETF stream it would be when we get the documentaction or the protocol action. That would be the time that the document enters our queue. We're not affiliated with IANA but I believe it's the same time when that approval message comes, the document enters the IANA queue also, so there's concurrent processing going on. Let's see. When we enter, when the document is entered into the database, when we get the approval message, it triggers a set of e-mails. And a set of actions. So we create a database entry oh that it gets entered into the queue. We create an actual hard copy of the file. Let's see. We go and retrieve the actual document, retrieve the author submitted source files if there are any. And for IETF related documents, send a notice of receipt to the IESG secretariat. All of the messages that get send to the are cc to the relevant parties, so depending on which stream it came from, it would be ccd To the appropriate represent tivt. So IETF, ADs and working group chairs. And potentially if anybody notifies us to please cc these people on these messages. [slide 30] Next one. So we're moving into the edit state. And the only thing I want to say here, is like, Alice, maybe you can come up and talk about the edit portion. ALICE HAGENS: So we call edit state the state when we actually read the document. There are lots of things that happen during edit state. Like reformatting the internet draft to look like an RFC, actually copy editing, and updating the soft copy with all the edits or the edits that are necessary. And asking questions, you know, interacting with the authors about unclear text, or any questions we have. So I'm going to walk through each of these. [slide 32] We start with two different kinds of source files. The NROFF source files are mark up that we can create from a plane text file. So if no source file has been submitted, by the authors, we create an NROFF source file. We also use XML source files that are used with XML to RFC a tool that's available on line. We only have an XML file if the author submitted an XML file. We don't create XML files. XML files save time generally speaking, because the formatting is automatically done by the tool. But we put a note on here to note the exception, not everyone's XML file is as clean as or takes advantage of the functionality of the tool, so we do spend time cleaning up XML files or or making XML files take advantage of the tool. So this example was in the references section, someone could just hard code it, just basically copy and paste in what they wanted for the reference entry, and they're not making use of the citation libraries or they're not even putting in a reference element, which would be the right XML element to use there. So the old thing would be this, they put it in as a figure basically. And we would go in and replace that with a include statement. And this is a P I that they've made a processing instruction specifically for use with XML to RFC and you're automatically pulling reference information for all RFCs or internet drafts. That's just a small example. When we do use XML file as you'll see later as we walk through the process, we eventually make an nroff source file. That's the official archive source format. We archive the XML file as well, but the NROFF is the primary. And we have different, from these source files we want to create an ASCII text file and there are two different methods, depending on the source file. So as I mentioned for XML we would use XML2RFC and you can set it to output text. For nroff we use our own script, which is called make RFC which we'll demo later. PAUL HOFFMAN: Two questions, first one is on what you just said, if you you have XML input, at the end, you actually generate text from XML2RFC or do you generate NROFF and then go to the last step? ALICE HAGENS: NROFF. PAUL HOFFMAN: Sorry I thought you said you generate text. ALICE HAGENS: We generate Text long as we can all through the process, at the very end, the tool that's available, XML2RFC, you can set it to do NROFF output and we set it to do NROFF output and might do final page breaking. PAUL HOFFMAN: Do you have to diddle the NROFF at all. ALICE HAGENS: We like to. There are some things to more easily control. BOB BRADEN: It's not a markup Language so it does not control page spacing very well. PAUL HOFFMAN: But the NROFF that comes out of XML, is not SANDY GINOZA: The tool right now is not set up to handle the 5378. It doesn't convert that instruction into the NROFF. ALICE HAGENS: The experimental version for NROFF output. SANDY GINOZA: So it creates the text but not the NROFF. So we are actually required to do a post edit. PAUL HOFFMAN: Okay. BOB BRADEN: One of the problems here is that the particularly the IPR stuff keeps changing, and we have to track it. And the tools people aren't necessarily, since they're volunteers, are not necessarily able to keep up. And so we have to make sure that the right thing happens. And we would like to have it automated but it's not. PAUL HOFFMAN: My other question was, because you, you know, like you you said, if it comes in in XML, you use that through the process. About what percentage these days are coming in with XML? Do you have any feeling or is that bursty too? ALICE HAGENS: I have a feeling it's increasing over time. PAUL HOFFMAN: Where is it now, give or take? ALICE HAGENS: So it would be just be a feeling, I don't have the numbers, but I'd say 50 percent. Many plus. Sandy? SANDY GINOZA: I probably would have said 60 percent plus, but it's a significant number. NEW SPEAKER: I didn't hear the number. What was the percentage. ALICE HAGENS: 50 to 60 percent of the documents coming in have XML files. NEW SPEAKER: Thank you. [slide 33] ALICE HAGENS: So, convert as I mentioned converting internet draft to the RFC format and you've seen what an RFC looks like, it has a running header and footer it has a specific top header block. Status of the memo it's usually saying if it's standard, informational, experimental. Other boiler plate like the copyright. And we also take a look at how the sections appear. For example, people often even include a paragraph or sentence about RFC 2119 requirement key words and we prefer that that's included in the text, not something floating around by the abstract. But we would move that to be sectione point one or, you know, and we also like to start the document with an introduction. So if they've put terminology one and introduction two, we probably move the introduction to the top. NEW SPEAKER: That's all in your style guide. ALICE HAGENS: Correct. We might not explicitly call out the case of the 2119 key words that I just said. SANDY GINOZA: But I think it does list the preferred ordering of the sections and that's definitely not first. NEW SPEAKER: Yes. Because I did read it. ALICE HAGENS: So, just a note on this one, about the future of the RFC editor, there's the document right now from the IAB about headers and boiler plates, and that will potentially be changing the status of this memo to be very specific depending on the source of the RFC. So, right now, we have 5 options for status of this memo, based on the different categories that have, that we'll talk about, or that actually Bob talked about the different categories. In the future, there are 19 options. So, knowing which one to put makes the step a little more complex. But it is something that you can imagine being automated with XML2RFC where it would know, you would put in what kind of RFC you have and it would put the right text. [slide 34] So copy editing is the actual mark up of the hard copy. SANDY GINOZA: So we have a sample here, one of the copy editors that we've had started a copy editing this document so we just want to pass it around so you can take a look at what kind of things she marks up. This in no way actually represents what makes it into the document. We recommend they mark everything possible and we make a decision when we go through it again. But if you guys want to take look, you can pass that around. ALICE HAGENS: So clarity, consistency, checking the cross references to section numbers are right, that, if abbreviations need to be expanded, flagging potential questions and trying to provide suggested text. We have tried copy editing with electronic format like PDF or track changes in Word, but when it comes down pen to paper seems to be the best option for copy editing, mark up, especially as we're trying to encourage somebody to read every single word, and to mark anything that appears to be a discrepancy. They don't have to confirm that it's a discrepancy, because it's going to be much easier for the next person, who is work WG the soft copy to actually do the search. And, determine if the term is appearing, you know, inconsistently within the document or that sort of thing. [slide 35] So, work WG the source files, we do want to make sure when we have an author submitted source file that it creates the text of the approved internet draft. So, this would mean, running the submitted XML file in XML2RFC and making a diff file between the text that's produced and the internet draft and seeing if there are any changes. If there are changes then it falls to us to determine why these changes are here and who we need to bring them, to bring to their attention, to whom to bring to their attention. [slide 36] So, for example, they could need changes that are, that we've seen in the RFC editor note, which is on the initial approval message, the protocol action, the document action, can have a note from the I go that says IESG editor note, and it's usually old, new. If the author has seen that feedback, which they probably have, because they're waiting to see how the IESG moves forward with their document, they might have already updated the document and give us the source file that's already updated, so if it's the RFC editor note that is the change that we're seeing there, they've already made a change that's been approved by the IESG. However, the other category, they decided they wanted to clean it up a little more, they changed their mind on a technical issue, any change that appears to be made to the internet draft that was approved, that's not part of the RFC editor notes, it's up to the RFC editor to make sure that gets reviewed. So we would bring it to the attention the area directors. Especially in a case of technical changes. And we would err on the side of assuming it's a technical change. [slide 37] This is an ongoing theme throughout the presentation and demo, we're evaluating changes made to the document to see if they are changing the technical content. Because at the poyment the document comes to the RFC editor, I'm speaking about the IETF stream generally now, it's already gone through a long process, either in the working group or for an individual submission the author has gone back and forth with the sponsoring AD so the document should not be going through, they shouldn't be tweaking their technical con tenlt and if they're trying to slip in changes through the RFC editor, it's our job to bring it to the area directors attention and ask them to confirm the changes, usually the author khims in and gives rationale why they made the changes. Was there a question, sorry? NEW SPEAKER: Well, if the author makes the changes, how do they get the document to make the changes? ALICE HAGENS: So we're still at the beginning of the process. So I'm talking about the initial source file; there are two ways basically that we're going to get an XML file which Sandy mentioned one is through if ID submission tool, so they've up loaded it. And they could have up loaded a file that doesn't exactly create the internet draft. Or, they e-mail us XML file and they could e-mail us an XML file that doesn't exactly create the internet draft. So they might have updated something. Another case we run into is, post processing, so for example, they send us an XML file and it almost makes internet draft but they've cleaned up the formatting somehow. So what they've done is created a text file from XML to RFC and messed with it and then submitted that to the ID. So then it's our job to figure out, if that formatting change that they've made it look better, you know, replacing a table or whatever it was, you know, I'm getting into a detailed case here, but basically, we're looking for discrepancies between the approved ID, and the text created by the source file that they submit, to make sure that A, they haven't slipped in changes that are technical in nature, and B, to see if there's something we need to re format or fix. NEW SPEAKER: I see that happen. PAUL HOFFMAN: Do you ever say no on the XML that they give in, if it has technical changes are or is this always that you deal with the ADor whomever. ALICE HAGENS: AD with the author ccd On there. So everybody, like Sandy mentioned everybody is on the e-mails. So, potentially the working group chair would chime in and say, this looks right to me but it has to go back to the working group. Or the area director, the most common case is the area director says, okay. Looks right. PAUL HOFFMAN: So, but you always accept the XML that the author sends you you. You never say, gosh, that's changed. We're going to start again, go back to the last draft we got and as you said before, you know, NROFF it? ALICE HAGENS: Not for changes introduced. I don't think. I could imagine the case where the XML file was so broken, that we start from scratch and do the NROFF. Because you can spend a long time going through, if they use square brackets through the whole things instead of X refs and that sort of thing. So once you get the hard copy from the copy editor or maybe it's the same person who copy edits it and does the editing, you know, the soft copy edit. You're reviewing the marks made on the internet draft, and deciding which of those edits should actually go into the soft copy. Some of this is RFC are or IETF style, some of this is also, and by RFC style, we could mean light editing, for example, if the copy editors marked every single instance of passive voice to change it to active voice, we're not going to do it. If they've changed every instance of i.E. To be such as, you know, that is, we're not going to do it. There's some things that you would do for more literally or, you know, if you had a heavier hand but at this point, like I mentioned the document is supposed to have been agreed upon text as a whole, to a certain extent, so we're not trying to force it to fit into a certain style. ALICE HAGENS: There are IESG notes that are also on the protocol or action document action message. These usual leevr for independent submissions, usually a paragraph and it goes on the front of the document. The RFC editor notes as I mentioned before, the old and new text, usually changes at the area directors have noted at approval. Other RFC specific knowledge and example would be language indicating obsoletes or updates relationships. So if they throw a sentence into the document that says this document replaces RFC da da da, and there's no mention anywhere on the header or elsewhere in the document that it's an obsolete, very specific relationship with an old RFC, then it's up to the RFC editor to bring that to the attention, again, of the same group of people and question if this document should be marked obsolete or you know what the purpose of this text was. Also, at this time, the editor is flagging IANA actions that will be handled during the next state, RFC editor state. Flagging any code components, this would be a MIB, ABNF or XML, it's going to be checked at the later state. And there's an extensive checking of references here. So in the references section every mention of an RFC is checked to see if it's obsoleted. If it has, then we're usually going to query the author, this document has been obsoleted, can your text be updated to refer to the newest RFC on the topic. Both the old RFC and old, and that would fall into the type of thing that was on the previous slide. RFC specific. So you have some feeling for the context of why, you know, why in this document is it okay that they're mentioning all the old documents and that sort of thing. But generally the idea is the most recent RFC on the topic. We're checking that the URLs work and we're trying to do some evaluation of whether it's stable. As you know, that's a hard thing to determine. But basically avoiding a personal web site. Generally, we have, if there are URLs in the document, there's usually one that's already, since the time the ID was approved, it's already broken. And that would fall into the category of a question to the author, to get an updated URL. So the presence of internet drafts in the references section in RFCs is a whole story unto itself. So if there are internet drafts that are references as normative references then the document is going to be held for those and they're going to move through the publication process together. Because by normative reference we mean one that's essential to understand the document. So the idea is, you can't publish that until that essential information is also available. So, any internet drafts that are in the normative references section are eventually going to be changed to RFC numbers. Internet drafts in the informative references section, we change to remove the full draft string and we reference them as work in progress. No matter the state of internet draft, whether dead or expired. This monthly policy has come under review several times. But internet drafts, in the references section are common. There are different ways of dealing with them, depending on whether they're normative or in informative. We also check to see if they've been published because it could be that there's a whole list of internet drafts in the references section but by now they're all RFCs, especially in the case where we have a document in our queue where it's been there since 2006. By the time we edit it, it turns out they're all published. That's the ideal case, because you're reference, because it's easy to reference. PAUL HOFFMAN: Alice, quick question. Do you have tools for doing that? Or is that pretty much wet ware? ALICE HAGENS: For determining which IDs have been published as RFCs? PAUL HOFFMAN: As well as what to look for. It sounds like you have a long list of gotcha's, do you have tools for that or pretty much your knowledge. ALICE HAGENS: It's pretty manual. To plug the tools that IETF, that work, though, the HTML versions, because they link to the full draft, you can just go down the list and click on the drafts strings and it will show you which one have been RFCs, but the more formal way to do it would be to go to the ID tracker and check and see if it's RFC published. And then we pull our bibliographic information or plain text, and if we're copying and pasting and if it's an XML source file we're pulling the reference from the citation library. PAUL HOFFMAN: Okay. Had been at this [slide 39] ALICE HAGENS: This is unclear text, various other reference problems, URLs and then we wait for the author to reply and update the document. Some examples, of questions. So if we're, if we're guessing what the meaning is, then we want to send ilt along. For example, all addresses or published in DNS and hence do not operate a two faced DNS. What? So, we can guess what the right thing is, and we always try, almost always try to guess, unless it's too broken to even guess. And, that way, the reply from the authors is more focused. Without a suggested text, they don't realize what's confusing. So all addresses are published in DNS and hence, you know, the author re nrid the site is does not operate to DNS. BOB BRADEN: Notice the ambiguity in the original. That was found as a result of this, apparently syntactic. Problem. ALICE HAGENS: This is repetitive text and we're usually not going through and cutting all repetitive text. Because throws a certain cadence to internet drafts where there's a value of repeating the text and make things clear in writing requirements. But in some cases, this is why we send it as an author query here. Willing to use U L.A. Address space. U L.A. Unique, wishes to use U L.A. s, You know, the repetition of which is to use, was basically what the question is about. If you've already stated at the top. We also spend a lot of time looking at lists, so if there's a way to make the list items parallel or state something at the beginning that doesn't have to be state aed with each list item, then we want to do it. NEW SPEAKER: I think I'm the author of that text. PAUL HOFFMAN: I didn't turn around. But you certainly should have reviewed it if you were not the author. NEW SPEAKER: Now, do you have any editing tools other than your editor? ALICE HAGENS: We have some scripts that we use to check for various things. [slide 41] So, for example, this checklist is kind of the overview. This is what the primary editor, the person who is actually inserting edits into the soft copy is using as a guide. It's not comprehensive but it does help you remember what you're supposed to be looking at. You should have gotten a copy of this, paper copy when you came in. This is the top or the left side, I guess. So, while editing, we make sure that each, all these it attempts have been reviewed. And during the demo, we'll walk through the bottom right here, CkText, dupewords and dotblank I, I spes and check text I'll talk about these, three scripts basically. The ispell is just spell check. But, cktext is making sure that there are not any lines that over 72 characters and also checking for appearance of TBD or TBA, which authors often use as a place holder to indicate that a number needs to be inserted that is an IANA assigned value. So we don't want to publish a document that has TBA Or TBD in it. It's part of our job to insert the appropriate values. dup words is checking for duplicate words, like the repetition of the word the happens a lot. Dotblank is checking for two spaces after every period. And, hanging hyphens. So, hanging hyphen, a hyphen at the end of a line of text is okay. Assuming the word naturally contains the hyphen. We don't break words that are not already having a hyphen. But, this will be a little more clear with the demo. NEW SPEAKER: Do you have anything that might, or is there any standardized text that you use that would call out obsolete words, or you know, the vernacular changes and say the use of the word button versus icon, versus, does anybody check for those things? ALICE HAGENS: No. So, there's no automated check for that. It's what the copy editor and the primary editor have marked or looked for. So typically what would happen for that sort of case is the copy editors is reading along and noticing switching back and forth between button and icon p and say button Icon, same, question mark, are or whatever their note is. And then the primary editor would get that and search and realize, yes, it does seem like they're using these interchangeably, if not sure if it's supposed to be made consist or not. Then it is would be AQ, author query. Are these supposed to be uniform throughout the document but because each internet internet drafts that we receive, it would be a hard to write a script to check for all words that are supposed to appear certain way. But we do maintain a terms list. It's a place you go and say, now they're spelling pseudo wire with a hyphen. And pseudo wire, it's an agreed upon we've heard from an area director or working group chair that they want pseudo wire to always appear one word and lower case. So we're trying to note those things, there's not, these lists are available on line. So from our main web page, there's a link for the style guide and it has a terms list and also an abbreviations list, which is similar type of deal. A guide to go to and say, you know, how have we expanded this abbreviation in the past, we want to do the same thing. So the big thing that comes into play with edit state is maintaining consistency within the document, primarily. That's the primary focus. Then if there's a set of documents that you're working on, things that are going to be published together and that are on similar subject matter and are probably linksd by normative references, we're trying to maintain consistency among the set. And then after the set, we're main tag consistency among the series. That's when you you end up really looking at your terms list and abbreviation list and searching in previous RFCs, especially ones that are normatively references. And then the final thing, yes, I guess that's it. The document, oh, the top IAB, within a topic, within the set, within the topic, like other documents, that would be the normative references or other documents that have a similar, or are about the same protocol, you want to use the same appearance of terms and abbreviations and then series wide is a harder trick because it's 40 years old, so you can probably find an old RFC that uses the term spelled differently. NEW SPEAKER: Well, right. ALICE HAGENS: In some cases. So, making, the long answer to the short question, we don't have an automated check for a specific terms or things that always have to be a certain way, but there is some institutional knowledge of which ones are recurring problems. NEW SPEAKER: Because I've used an editing tool called Acrocheck, which you you can teach it rules. And you set it up, and for whatever the current terms are, of course, you wouldn't go way back. ALICE HAGENS: Right. NEW SPEAKER: And it takes care of all of those kind of thing that's really helpful. It's called Acrocheck. SANDY GINOZA: It is hard, because a lot of set rules, a lot of the authors want to change it back. NEW SPEAKER: There's that element also. I've used the too bad approach. But that doesn't always sit so well. PAUL HOFFMAN: Well, to go on to that, Sandy, just what happens when you send an edit back, saying we've done this consistently, because it was an RFC so and so, and the AD goes great and the author says, I hate it. Do you have a priority for, of the people looking at the reviews on who wins. SANDY GINOZA: IP typically it's the author. But usually the AD and author have to come to some conclusion. We tell them, we don't engage it in that. We tell them to work it out. ALICE HAGENS: Part of the way the process is, we ask a lot of questions earlier, and so some of the more terminology questions or things that would have been changed throughout the document, have been discussed. So the editor knows already, and has reached like say, reached a decision with the area director. This is also on the checklist, marking the formal language, just flagging it to be checked at the next state and extensive checking of the references. Sometimes there's one big references section. The nature of RFCs is you have to split it into normative and informative. And it provides guidance to the reader on what's really essential. These, I mentioned. Matchref is a script that make sure that every reference that appears in the references section is cited in the text and vice versa. And checking that the URLs work. The draft, so drafts colon, it's kind of a floating things. As I mentioned for internet drafts there's different things to take care of. So, basically, if I were the primary editor, I would give the file to go to the next step. Give the file to Sandy for check. RAY PELLETIER: We're going to need about a ten minute break at some point in time. Is it at if RFC editor state or the auth 48 state do you recommend? SANDY GINOZA: We can do it now. It can be a document as, the editor believes it's ready for publication as possible. (Recess.) >> SANDY GINOZA: So Alice was primary editor for this document. We're now going to move it to the RFC editor state until those issues have been resolved, so that's a holding point for us. So, at this stage, we want to go through and assign the RFC number, and if there are any sub series number, we're going to assign that one also. Throws a little bit of logic involved here for sets of documents, knowing which document comes first in the set, how they should be numbered sequentially, and for obsoleting documents or updating documents, we want to sort of use that same logic. Let's see here. We also want to, this is where, the first time we're going to go through and actually assign the IANA assignments. So at some point, IANA has sent an e-mail saying we registered these values, these are the actions that have been taken. They'll send a URL where they're locate indicated. We're going to go through the IANA consideration section and make sure the text matches up. If they have TBA Or TBD, we insert the value. And sometimes the values don't match what IANA has assigned probably because the values have been assigned somewhere else and we need to up indicate that text as well. We tend to change the text to past tense because the actions have now been completed. And this is what I was talking about the descriptive text and consistency with nups. Sometimes it's not just the IANA situation, sometimes it's fill in this IANA number and value, and we want to make sure we catch all of those before publication. And, we're going to check that the correct boiler plate material has been applied so the correct status of the memo, the correct boiler plate, so the 5378 copyright and is the 6c3 paragraph necessary for this document. The other things we want to do is check any editor marked flags, let's say Alice is working on the document and she's really not sure what to do about this, maybe you will have a better idea. So I'm going to take a look at those. Make sure that the RFC editor notes that Alice mentioned comes m the document actions, the approved messages, make sure that those got inserted properly and completely. Update any references, so that's particularly helpful for sets of documents, so if let's say there is a set of 5 documents that are moving through the queue at once, they're going to be assigned RFC numbers at once, they all reference each other's. All the references need to reflect the new document numbers. We're going to create a diff file so now I want to compare the original file to the edited file that Alice has produced. We'll show you later what that looks like. But it's just to help us identify the changes that have been made, possibly looking at it a little more closely and seeing if we think that that change that might affect the technical meaning, possibly reverting some or adding some of the ..... We might decide this change is a little technical and I'm going to take this change out and send it as an author query. It's a consistent training tool for us. So, whoever is looking at this document at this stage has the opportunity to see what kind of marks the person before them is making. And I might notice some patterns or things that we don't think we should be changing and then we have meetings at this at some point, what should we do in these cases, can we make a default rule? Can we update any of our files to reflect this so we don't have to do the same research the next time this comes up. So after I've gone through and if I have to make any changes, I'm going to make all my changes m the source file, re makes the text files and then I'm going to perform, re could the checks, the automated checks, because while I'm in that file, there's potential for me to change something unintentionally. So these are the formal languages that we currently check. MIB's, XML ABNF. And the internal ones we use an internal directory of MIBs, because they can pe attentionly draw off of other MIBs in that directory. With MIBs comes along some standard boiler plate text that is not common to all documents. They have a special security guidelines section and a boiler plate, like introductory paragraph. It's standard, it was agreed upon. We don't do it. XML, we check that it's well formed so we use XML lint for that. And for ABNF there's an on line checker that Bill has created and it's located at this site at this location. So, we've, because we do this with pretty, with some frequency, we've been able to see what kind of errors are generated and we might be able to troubleshoot the error beforehand. So we can send the authors an e-mail saying, it produced these errors, we were able to fix it doing this, is this correct. And see what they say. The other thing to note about these, when these appear in documents, each one has to be extracted from the document and included in their own file before checking. So, at this point, we've done all of the final checking, we think the document is pretty ready for publication. We're going to move it to the AUTH48 state, and this is AUTH48 state. And it's posted and it's publicly available to retrieve. If there are, it's basically the point, the last chance for the authors to review the documents and suggest any changes or tell us that you changed the meaning here, please revert it. We don't like this change, it's just an opportunity for them to make any final clarification. Hopefully they're editorial. NEW SPEAKER: And do most of them look at the end result? SANDY GINOZA: They have to. We require that each author that is listed approve the document for publication. BOB BRADEN: It doesn't mean they'll read it. SANDY GINOZA: True. NEW SPEAKER: That's what I was getting at. BOB BRADEN: We do the best we can. SANDY GINOZA: So at this point, send out an auth 48 message. I give the folder back to Alice who handled the primary editing with the document because she's the most familiar with the text. NEW SPEAKER: On the auth 48 is what the author gets is actually the edited document and then another pointer to an HTML different diff thing so you can see the changes very clearly. [Slide 48] ALICE HAGENS: Sorry. I forgot some slides here. So, exactly what Bob was saying, we want to create the final text and diff files and we're going to post these in the public repository, update our database, we're going to change the state, to auth 48 and we are going to include the RFC number here, and we're going to create and send an auth 48 announcement which again gets distributed to the authors, area directors, any other relevant parties, and send the authors any remaining questions. A couple of those questions might be for example, we talked about this earlier, let's say Alice flagged the document and she was like, do you have any idea of what this text is trying to say. And I might read it and say, no, I don't have any better idea than you. And, send the authors another question. PAUL HOFFMAN: So there are two times that you're sending authors questions. You might do it during the earlier processing anyway, but you also might ask similar questions here. ALICE HAGENS: Right. PAUL HOFFMAN: In the auth 48 step. SANDY GINOZA: Right. Right. This is an example of a question we just did ask the authors of this document. During the RFC editor state we found a discrepancy in the IANA repository with what we think should be in the IANA repository. So this here is the actual text that was in the document. And if you notice, for these two object identities the description is the same. So we're asking -- BOB BRADEN: How did you ever find that? SANDY GINOZA: So the question here was, is this intentional, is this correct and the author re nrid, no, the first one should be this. PAUL HOFFMAN: Did you find that or did the tool find it? SANDY GINOZA: We found it. ALICE HAGENS: This example is really a discrepancy in the IANA MIB in what IANA has posted as opposed to a discrepancy with IANA, which is a different thing is more common, where what you have in the document you're editing doesn't match what's onto IANA site. So, so, there are different kinds of IANA mismatches. SANDY GINOZA: Right. This is just one example. ALICE HAGENS: But that's the type of thing, even if were not IANA text, if it was a regular MIB and we spotted the descriptions were the same for two different objects, it would be an author question. In this case, it's more complicated because if the author says, right, you know, that needs to be fixed, then our job gets doubled, because we're not only fixing the document, then we're contacting IANA and asking them to update the posted information. PAUL HOFFMAN: Quick question, on the auth 48 are you also sending that to IANA or have they pretty much seen everything that you believe you're saying in the IANA considerations, before auth 48? SANDY GINOZA: I don't think I understood that. PAUL HOFFMAN: Is IANA involved in auth 48 also or not. SANDY GINOZA: No. They're not. They're not ccd on this dock: This is our time to interact with the authors. If it comes up that there's discrepancies, we may contact IANA and involve them at that point but we don't involve them, we're probably see if it's an error first and if we don't know how to resolve it, we cc all the parties and say, let us know what needs to be changed in our document or the IANA web site. PAUL HOFFMAN: Do you send IANA the text of the IANA considerations, sort of when you you think it's all done? Or -- SANDY GINOZA: No. What happens is, we edit the document, and they send us the e-mail saying, IANA completed, action completed according to what's in the ID or according to whatever then the author have interacted. I'm not an IANA representative, so my best understanding is that the authors themselves have to sign off on the actions that have been xleetd ted. So they are themselves or they request it. So they are themselves saying that yes, we believe this is correct. Whether they actually do it, I believe it's the same as our auth 48 process. I don't know how closely they pay attention to it. PAUL HOFFMAN: I'm not sure IANA asks for a sign off. Irrelevant for here. Sorry. SANDY GINOZA, then at the auth 48 stage if there are changes that we need them to make, we'll notify them. Depending, well, they'll see this in later slides, but depending on how large those changes are, we may want confirmation that, their registries have been updated before we publish the document. Or there may be minor changes like a spelling error or something like that where we would send that at a later time with the RFC number and the publication date so they can update the references. Does that answer your question. PAUL HOFFMAN: Definitely. Yes. Thank you. SANDY GINOZA: This is an example of some of the questions and this is where I would hand Alice back the file for auth 48 processing. ALICE HAGENS: So at this point, Sandy sent the e-mail with the links to the edited document and the diff file. If they submitted an XML source file, there's also a link to the edited XML source. And she's also sent any questions, any additional questions. So at this point, I am waiting to hear back from authors. The easiest case is they say, okay, good to go. Assuming there were no questions. Sometimes we get, okay, good to go and they haven't answered the question. And then good to go except could you answer the questions. But people send changes at this time. So, back to the same thing mentioned earlier, of evaluating if the changes they sent are editorial or on technical. And if they're technical then requesting AD approval. This can get complicated. For example, if an author sends 20 changes, and we think three of those changes are potentially technical content, then we somehow are drawing the AD's attention to specifically three changes. Usually, that is easy enough with section numbers. You know, or diff file that shows only the changes since the originally posted auth 48 version. Another option is, just taking the text from the e-mail from the author, so these three changes from the author. We also edit any text that they send in those changes, so if they say, replace with this paragraph, we want to read this paragraph, the same as if it it were in the original internet draft. And we want to make sure whatever rules we've applied to the rest of the document get applied to that paragraph too. So terminology, et cetera. We want to make sure when they send changes they're not editing the RFC editor note. So, if we've edited something one way specifically because the area director has asked us to and we receive a change from the author, there's no mark on the text, right? I mean, theoretically, in the source file, there could be a comment that you could have flagged, this text is DNE, do not edit. But we don't do that. So, the way that you would know that this text is do not edit is by having the document action or approval, approval, protocol action, message that actually lists the RFC editor note and make sure they're not changing that text. That would also apply to the MIB boiler plate that Sandy mentioned earlier. And it would also apply to the IESG note. Occasionally an author sees the text of the note the first time when it goes to auth 48 and they say, hey, I don't want this on the my doc. Well, take that up with the IESG because the RFC editor did not produce that text and we definitely can't change it. We would repeat the same process that Sandy did, as part of RFC post the document. So we're editing the source file, remaining the text file and posting the files again. We're keeping track if the document has 5 authors, you know, it gets, it can be a lot to track. For one author, it's usually pretty straightforward. Hopefully, you don't get into a lot of iterations. You, they send one set of change and then another set. Typically people send all their changes at once. If there are 5 authors and they don't get together, then you're receiving 5 people's changes and making sure that they're not changing each other's text. But usually they're pretty, at least cognizant of what their co-authors have sent. If we have escalated to the area director to check on technical changes and then we're also tracking AD approvals. If we haven't heard from the authors, we're sending weekly reminder messages, we don't believe we've heard from you regarding this document's readiness for publication as an RFC. Sometimes, not the first time they see the message for various reasons but we usually send those reminders from our personal accounts, as opposed to RFC editor.Org, depending on people's mail filters, what they actually see. So, we found the reminder messages are a grood way to actually get responses. So, we gave a you copy of the auth 48 checklist. It's basically the author names and whether they've signed off or not. But it is also includes RFC number, sub series number, and we note the month and year of publication. Any notes that I would want to pass along, so, for example, if at auth 48 Sandy had send along a few more questions, she might write on notes here, like AQs so I know when I'm doing the auth 48 process, I'm waiting for questions, which I should know anyway, because I'm traiking the mail for this RFC number. Also, my initials are in the subject line of any mails, that are in reply to the auth 48 notification message. RAY PELLETIER: Did you define what the 48 stood for? ALICE HAGENS: It's an optimistic time line for completing this process, is 48 hours. SANDY GINOZA: I want to say, historically, it was actual 48 hours, and if there was no response from the authors within 48 hours, go to publish. PAUL HOFFMAN: That preceded you, Ray. RAY PELLETIER: By years. ALICE HAGENS: That preceded me. PAUL HOFFMAN: It was a scary time for us authors, believe me. SANDY GINOZA: Yes, so the name has stuck around but that's the minimum amount of time that we wait before BOB BRADEN: It's unfortunately, more often 48 days. ALICE HAGENS: So as mentioned if the source file is XML, at the very end of this, after I've gotten all the other sign offs, I'm going to create an in NROFF site. BOB BRADEN: The really unpleasant situation is when the author decides to update the document when it's at the auth 48 state. Because the new version is presumably more correct and we want to publish the best version, on the other hand, that breaks the process to pieces. And, we've had words with some authors. RAY PELLETIER: Well, updates that are technical in nature winds up going back to the area director and the working group, right? BOB BRADEN: Actually, the difficult cases, independent submissions, they come back to me, or to the reviewer. Because the whole process starts over again. PAUL HOFFMAN: IETF drafts can't be revised once they've been sent to you; is that correct. SANDY GINOZA: They can, but we would send it to the ADs and ask them what's going on. Sometimes they withdraw the document are the queue and say it's not ready yet. PAUL HOFFMAN: Got it. So there might be a reset for you folks, in that case where the (Several people talking.) SANDY GINOZA: Right. PAUL HOFFMAN: We'll do it again. SANDY GINOZA: Would have already invested this time in this version. NEW SPEAKER: So at what point can they not withdraw it. BOB BRADEN: The day we publish it. NEW SPEAKER: So all this work is done. Forget it. From end to end, how long is the normal document? SANDY GINOZA: It varies a lot. We'll actually show. do you want to show the summary now. ALICE HAGENS: This is for the queue. SANDY GINOZA: The current queue right now. ALICE HAGENS: So on average, Edit, RFC edit. Auth 4, auth 48 has a long average now. Because they're waiting for copyright issue to be vee solved. So three point six is more representative. But so, you're looking at a month. Then things get more complicated for the IANA holds, ref holds longer. SANDY GINOZA: I don't know if you know what all of these things mean. These are the states that they can be in our in our queue. Misref, I think Alice explained this earlier. But if there is a normative reference to another document and that reference has not yet arrived in our queue, we can't publish it yet. So we move that into misref, which also includes documents, for example, second, third generation misrefs so let's say we have two documents in the queue, a references B. B references C. C is not here yet. They're both in misref, NEW SPEAKER: Then who is managing that to make sure. SANDY GINOZA: We are. I NEW SPEAKER: Is there some automated way. SANDY GINOZA: So, this one helps us a great deal. RAY PELLETIER: Cluster management. SANDY GINOZA: Right. This is a list that gets produced weekly for us, automatically. So, when sets get released, so, let's take a look at misref state here. They're all grouped by state. So, sorry I can't make the screen wide enough to show this with proper formatting. All these docs are in misrefs, when the set is complete, the whole set will then pop up, theoretically, if everything is working properly, into the edit state. So when we look at it weekly, we say, okay, this set has been released throughout those documents. The other thing for us, because we've been here, we try to pair those documents together as they come in. Not so easy if you are not in the same location. But, somehow, will we should notice that the documents are being released. RAY PELLETIER: Are those also shown in the clusters. Can you show that there? SANDY GINOZA: Sure. So this is our RFC editor queue. Recently, we added this view here. Which shows all the various clusters. And by cluster, we're talking about any documents that are related by reference. These may not necessarily be solely normative references in this list, because sometimes the AD or the authors might say, this document is not a normative reference but we would would really like for this document to be published together and we're going to note that here, because they cannot move forward. RAY PELLETIER: So looking at cluster three for example, which one is holding up. -- SANDY GINOZA: SANDY GINOZA: This one here that says not received. All these other documents are in the queue somewhere in the various states. The first two have been published, they are just referenced by one of these documents, so they're okay. It's not a big deal. Edit star R. That's our way of referencing star R. It's a reference that it's in the queue somewhere. Misref is, it actually has the reference to the document that's not arrived yet. And this not received one is the one that's holding up the entire document because it has not been received by the RFC editor. RAY PELLETIER: So now you interact with the IESG to see what can be done to accelerate the clusters. SANDY GINOZA: Kind of. The IESG receives the key summary weekly. We don't really send reminders. I know Russ sends reminders. RAY PELLETIER: Right, he does the cluster busting. BOB BRADEN: He does policing, right. SANDY GINOZA: He'll send reminder messages saying, can we do something to unblock these. ALICE HAGENS: To answer your question about automation, theoretically when that one document not received arrives, it is automated, this state changes happen. And like Sandy said, that other list that shows them altogether by state, they would flip to the top and be in edit state. But you can see that, well, actually not from this view, but if you click C3. So this gives more detailed information for each one. SANDY GINOZA: For the particular clusters. BOB BRADEN: This is a significant piece of programming that you paid for. And I think you you got your money's worth. RAY PELLETIER: It's a 2009 programming thing. ALICE HAGENS: The beauty of this, from what we're talking about before in edit state, so not only in document management way but in editing sense we want the terminology to be consistent in the set of documents that we're editing right now. And we also ideally want to match up with the documents that we published before. So if we're running into terminology questions, this is an east say way to say let's see what they did in the other two documents in the set. SANDY GINOZA: So it's possible those documents might be inconsist also and we have to make a decision what the correct terminology is, but it's good reference points. PAUL HOFFMAN: You you also have to cross reference the IANA sections, I have to say, embarrassingly, I was the progenerator of one of those, that was caught by you guys. PAUL HOFFMAN: The IANA considerations were duplicated but not completely. SANDY GINOZA: That's become pretty difficult, like when sometimes they have, we had one set recently, and it's, this one is sort of a special case, rare case, but it was terminology and they had the same terminology repeated in four different documents. And when you make the edit to one document, you have to make the edits to all the documents and the authors were sending us varied information. We don't like this definition, we want to use this definition. So sifting through all those e-mails and finding out what the final verdict is, makes it difficult to channel these sets. NEW SPEAKER: So if you were checking all 5 for instance, for consistent terminology, would you have to, you'd open them up individually, and -- SANDY GINOZA: It depends. My initial read, I'll probably flag all the inconsistencies I see within the one document. Then we might make one file to see if it exists in all the documents. Or it might be let's say, if I am read ting next document, I notice the same inconsistencies, we can do a search or whatever the case may be. ALICE HAGENS: If the set is big enough. If multiple editors are working on the set, like Sandy mentioned earlier, editorial meeting to sit down and talk about how we're going to make decisions on what we're going to send as author questions and/or make decisions. NEW SPEAKER: Would one editor have the whole set? Or it could be -- I mean, you you did say a number of editors. SANDY GINOZA: It could be broken up. It's just a matter of us trying to get our edits consist, in that set, and then who actually makes those edits doesn't necessarily matter, as long as we're in the same page about what edits are to be made. ALICE HAGENS: Which is part of the process with RFC editor state. Like the checking state is also the number assignment state. So at that point the documents are grouped because the numbers are assigned sequentially, probably. And then, they're also, the beauty of having one person do RFC editor check for maybe documents that had different primary editors, that's also a major consistency check point. BOB BRADEN: How close are you to the end. SANDY GINOZA: We have a few more slides to get through and then a demo. BOB BRADEN: Why don't you move on. SANDY GINOZA: Okay. SANDY GINOZA: So this is sort of what Paul's question was earlier. How we notify the IANA, how we interact with them. This is again, the first arrow, is if it's significant changes to the IANA registries, we want to make sure the changes have been implemented in the registry before we publish the document. So we're going to notify them earlier, before publication. For every document that has IANA actions, we need to send an e-mail saying this is the RFC number, publication date, if there is minor areas, small nits, we might send those corrections at that time. While you're updating the registry, correct this also. So, at this point, we pretty much are moving towards publication. We think all the authors have signed off on on the document, we think it's ready to go. So this is the point where we consider publication to start. I don't know that this is going to fit within the new model but we haven't tried to break up how this fits. So, let's say Alice has finished auth 48 and she says it's ready. So what I want to do now is it's possible I might need to change the month of publication, just if Alice has forgotten to do it or something. If auth 48 starts in one month but doesn't complete until the following month. Updating those and the necessary references in the same manner. This next step, next bullet point should already have been done. If the file is XML, this is again where we're going to convert it to NROFF to have an archive file. That's the official arc kivl file. Change the copyright and possibly re break pages and redo the table of contents, but this has been done before it makes its way back to me. I'm going to do re checks and see if there are any changes and just because they're easy to do. So before final publication, let's just check it is one more time. Again, cross checking T O C, table of contents, pagination. And I'm going to do a final formal language check just because text changes during auth 48. So it's possible that the language is now broken but we want to make sure that it still works. So, this is the bottom of that checklist we saw earlier. The final checks that we're going to run through. We've already seen these. We've done these before. We run them once again before publication. ip going to check off, yes, I checked the languages, they're good. And, if there were IANA actions, that's marked and I need to send IANA a message saying you can close your references. RAY PELLETIER: How long does this take? SANDY GINOZA: It depends how long the document is. It's relatively fast. I want to say, average ten minutes. ALICE HAGENS: It should be perfect at this point. SANDY GINOZA: Right, right. So, at this point, I'm going to update the database with the document information, update the records. So, if this, so for example, if the RFC that we're publishes obsoleted an old document, I also need to go into that entry in the database, and update the obsoleted field so that they are showing the same, the corresponding information. If there is sub series, if there is a sub series number affiliate ted with it, we need to create or update the existing entry for that. We're going to clean up the directories, move the RFCs into their final locations, notify IANA and notice fi them also if the registry needs to be updated. So, this is where we come into some of the scripts that are for publication. So we updated the index files using the web application. We're going to run the publication scripts, there's two sets actually, they do different things, but at the end of these two processes, all the files are in the correct place, available via the search engine. So what I would do after that is just check to make sure that the scripts did the right thing, they worked. The RFC is available on line. And then I'm going to send the publication announcement. Document has been made publicly available. There it is. And then, I just want to ensure that the announcement was distributed so make sure that it shows up in our mailbox, do some clean up of our own directories, and then we here at ISI actually have a hard file for each RFC that's been published. So the internet draft is in there, the final text of the draft is in there, any necessary e-mails that we think should be in that file are there, for example, the IANA actions, or some of the author questions, something like that. And we file those away. BOB BRADEN: We should talk about what to do with those paper files. I don't know that ISI wants to keep them. RAY PELLETIER: I have an outstanding action with Jon about that. And, BOB BRADEN: For closet space. RAY PELLETIER: And other things we talked about. Including software things, are closed. I think you you you had to do some checks on that and he has not yet resolved it. But we will resolve that. And I think you've got maybe fifteen filing cabinets, four drawers. RAY PELLETIER: I can't imagine they want to store them in the basement forever. BOB BRADEN: I think it goes to the RFC one. But I'm not sure. SANDY GINOZA: So here's where we're going to do the run through of the demo how all of this works. PAUL HOFFMAN: You're not going to use a draft that I wrote? SANDY GINOZA: I don't think so. RAY PELLETIER: Let's pick one of those. SANDY GINOZA: So, we have this document here, it's been here nor a long time. This one has been waiting since 2006. We picked it as random sample. So, we receive a document action. We're talking about the IETF, right, protocol action, dk can document action. RAY PELLETIER: Could you name the tools that you're using. SANDY GINOZA: Sure, this is our database. We call the it. RAY PELLETIER: Fred. ALICE HAGENS: It's P H P front end to the my SQL database. SANDY GINOZA: What I want to do is add a new document to the database. So these are the various fields that I knee to fill in. What I'm going to do actually is jump to the actual entry for this, because this document is in the queue. Okay. So this is all the data that we've put into the queue. Or, yes. Sorry, into the database. It holds the status, the source, the working group that produced it. The normative references. And so, once this document is submitted, we would see it show up here. RAY PELLETIER: Are these screens in the procedures manual? SANDY GINOZA: No. So this is where it would show up and it automatically detects that this document has not been received so it's going to designate it as misref BOB BRADEN: This is the standard queue. ALICE HAGENS: It references the document. SANDY GINOZA: Sorry. It references the document. At this point I would check to see if there's any IANA actions m the dwok ment and I would flag that in the database and I would also check for normative receive references add that to the database. Once it's been submitted and entered into the queue, it triggers a set of e-mails, to the authors, saying this document has been added. Do you have the source file, please provide it. We'll send an e-mail to the secretariat saying we received this file, here it is, it's been added to the queue. There is another one. BOB BRADEN: Basically every state change triggers an e-mail automatically. SANDY GINOZA: So, PAUL HOFFMAN: I know this is an older one, but you have the copy editor already signed there. Is that normally when you pick the copy editor. SANDY GINOZA: These fields only get completed when somebody taken the document. ALICE HAGENS: I do it now. If I pick up the folder in Sandy's office, and I open it up and I saw that it had already been copy edited so Kate Peterson, is our copy editor who you saw the red mark up on the that before. So she copied edit and then I put my initials. P E primary and C E copy editor. Often it's the same person. But it's nice when it's not the same person, because, it gives you an opportunity to look at the marks critically, and decide, you're going to look at your own marks critically as well, but it's a separation between doing the insert edit, the copy edit and insert edits. PAUL HOFFMAN: Some of these copy editors are remote, you said. Do you just Fed-Ex things around? SANDY GINOZA: Actually, so we only have one copy editor that works with us. And she actually is editing files via PDF and sending us PDF files. And then we do our own copy editing for a lot of the documents. It's just personal preference you you how you like to. PAUL HOFFMAN: But your remote copy editor is not Fed-Exing things to you. SANDY GINOZA: At one time they were. It depends what tools are available. But she said for her, a PDF, she was comfortable with it so that worked fine for us. RAY PELLETIER: Just as a footnote to that, as Kate is actually is under contract with ISOC, as a part-time copy editor. That ISI has access to. And, it's real part time. Sometimes it's 5 hours a week or ten hours a week. I don't think it's reached ten hours a week, actually. SANDY GINOZA: Probably not for sometime. RAY PELLETIER: And there's no intention that ISOC is going to have a field, you know, afield editors under contract like that. It's expected that the production center will have all the staff that they need. PAUL HOFFMAN: You said that specifically in the S O W that you will not. RAY PELLETIER: Make that very clear, that we're not going to do that. So, that footnote. BOB BRADEN: Kate has been resident here for a while. So she's absorbed the the rules. RAY PELLETIER: Has the culture. BOB BRADEN: Culture. PAUL HOFFMAN: Okay. ALICE HAGENS: So as we talked about, this is where I go through the hard copy, page by page and look at the copy editor marks and decide what to insert. But what I'm starting with, assuming they dent send this source file, what I'm starting with is this, the internet draft, the plane text of the internet draft. So I would end up inserting directives and once it was formatted, it would look like this. So, you know, NROFF commands create the headers, to center lines, it's just basic mark up. ABNF, inserting status of this memo and copyright notice. So, depending, it's subjective, depending on who is work WG the soft copy. Some people like to go through the whole document and re format it and then actually sit and do the insert edits. That's really only applicable for in R off. XML there's, you're not going through the whole document reformatting. There's just a few things to switch at the beginning of the XML file. So for this one, this is what the XML file would look like, and we use a subset of NROFF, so that the mark up is pretty straightforward. ALICE HAGENS: So the XML file looks different, because it doesn't look like the text output. PAUL HOFFMAN: This was one that you received as XML? ALICE HAGENS: This is one where I'm showing an NROFF and XML file. PAUL HOFFMAN: Okay. So is NROFF on appeared add the very end of this process. ALICE HAGENS: Yes. We NROFF this file and it's from 2006. It's good for a demo. Well, not too much of an ouch, PAUL HOFFMAN: NROFFing is not too big an ouch for you. ALICE HAGENS: Depending how long the document is. If it's 30 page document it's not that major. A hundred page document, yes. Or if something that contains a lot of lists, because NROFF formatting gets trickier and trickier. So I would insert edits into the source file. For example, here, I already made this edit actually. I added a serial comma, like they have a list, I put a comma in here. You know, you go through and you you make your changes. And then, depending on what kind of source file you're using, you're going to create the text file, so, for the NROFF file, I would run make RFC. So, you know, run this script, at this point we keep the, so make RFC is taking the NROFF file, outputting ASCII text. RAY PELLETIER: What system are you using? ALICE HAGENS: I am SSHd into NROFF. To the file system. So you can see this looks like an RFC, the text output. So if I was looking through this document, I would say, oh, there are only two lines of introduction on this page, I need to fix that in the N ROFF file. You can tell this not been fully edited because it still contains the draft string reference. For XML I, we use the web page. So I would go to the experimental version, because this little boiler plate and put in the XML file. I'm going to do window output so you can see that it produces the same thing. But if I were actually working on the document, I would have saved it as a file, not just display it in the window, unless I was just checking to see how things line up. So you can, and so, I basically would save this text file and start run ting automated checks, once I thought I had finished inserting all the edits and everything was good to go. So some of the checks that we talked about, check text is checking that there are no lines that are over 72 characters. Actually going to run check text one which is specific for before, it has RFC number inserted into ilt. And it's stopping me for, you can see here, it's stopping me nor TBA s, Suspicious TBD. The IANA value is missing, that's okay. Because I'm edit state right now and the IANA value is supposed to be missing. It's also telling me I have a line that's over 72 characters and telling me what the extra part is. So m the NROFF file, I would look at this and say, okay, this line is over 72 characters. We have a little yardstick that we can insert in here and see, but IAOC see what the problem is, right, because the output here, told me, so, I end up cutting some space here, and re run it, you know, put m the file back, re run it, and run the check again and see that this one has gone away. I'm going to mark on my checklist for the next person that gets the document, about the TBA And TBDs. Examples of other checks that we talked about. So, dup words is checking for duplicate words and it finds the. So, I go back into my source file and make the necessary correction, right, cut the xwra one more, two more automated checks. Dot blank is a more painful check. Looking at periods and hyphens, so, here it's stopping on for example, versus, here. Because versus, the period looks like the end of a scene ten it thinks that there needs to be another space there, because it's an RFC style rule at the end of every sentence, terminating period has two spaces after it. So, with this, use your brain to know what to ignore and what to act on. It's also stopping for every hyphen at the end of the line. These ugly ones, the I dash are from these references that have not yet been updated, the IDs for the internet draft. So, when the text wraps, you can see one of these up here had a back slash percent sign to make it not wrap. But actually, all those are going to be changed. So, et cetera, et cetera. So easily we find one where it actually was the end of a sentence, then I go back in here and add a space after the period. The last automated check I'm going to show is the one that checks the references. Let's say I've gone through and fixed all the referencees, match ref is going to make sure everything matches the citation in the text. It stops on anything that has a square brackets. So, this is also having problems with the long ones that are broken across the line. So I'm ignoring these e, if I were actually preparing this document for publication, these wouldn't be in here at all. But the way it works is, looking at the references section and saying, there's an informative reference and a citation, if it thinks something has appeared twice in the references section, it warns you that there's a duplicate reference. So basically, these are all okay. It's going to give sklam maition exclamation points. So when I run all the checks and check everything off on the checklist, I think the document is ready for RFC editor. I might have flagged some issues and I've definitely flagged the TBD and TBA On the hard copy. SANDY GINOZA: So this would be when she would give it to me and we would start the RFC editor check. And it's sort of -- ALICE HAGENS: I would change the statement database. I go to the database and change the to RFC editor and automatic state change message goes out on and then I give it to you. SANDY GINOZA: Okay. So. What we were looking at earlier, what we used to look at a diff file is H T M L W diff. It's just basically create the diff file and I wanted to show you what that looks like. So, this is the document that we were just looking at. And, this is showing you what the ID, what it was submitted as, and the marks crossed out is the text that has been dwee leet ted. The green is the new text that has been inserted. So it's more helpful, boiler plate always changes. The more helpful part for the authors, I believe is when you get into the text, and you can see right away, what's been changed. So this is the comma Alice inserted, this was the pseudo wires, lower casing it. This was one from the XML file, so that the references were updated here. So this is basically what I'm taking a look at to see if there's any consistency issues, if ti think there's anything we need to make decisions about, revert any text, ask the authors any additional questions. So, once I've gone through all of those, all these steps and I think the document is ready to be sent for auth 48, I'm going to go back to the database and I've gone moo the find document again, and this is the entry, I would change the state here. It would have been an RFC editor so I would be changing it to auth 48 so IAOC do that, just to show, because it's, the e-mail won't be triggerd yet until I create it. I would insert an RFC number. So, just for exam for example, we're going to submit it. It's going to give me a warning if it thinks I missed some fields and IAOC choose to commit it anyway. It's saying, yes, successful, all these things have been done. And what I want to do now is create the auth 48 e-mail. So this drop down box is a list of all the documents that are currently in the queue. So all I have to do is find this ID. So it's this one. This is just an example. Like sometimes those three above it have similar names, you have to make sure you're picking the correct document when you are creating your e-mails. So I want to select this one. So the auth 48 e-mails, there's two versions, different directives on how to submit your changes. So this one was done via XML so I'm going to create the XML message and here we'll see all of the authors that are listed, CV ccht ourselves so we get the message, the area directors, the working group chairs. And this is again, like Alice mentioned the initials of the primary editor, that the authors will be working at during auth 48, this is the first time the authors will see the RFC number and this is the body of the document which points to, points to the files where they're located and gives them the directions on how to update the document. So I'm going to close this because I don't want to actually send it out. And, the last thing I want to do here is show you like, I don't want to demonstrate all of the tools, I mean, or the formal language checking shs but one that we were going to demonstrate is MIBs, that's probably the one that takes the most training onto recognize some of the errors. We have the benefit of working with Burt wine nan for a while, who sort of helped us get some of this information down. So like I said, I'm going to go into the MIBs directory we host MIBs, so any RFC that has a MIB, hopefully there should be a file in the directory so any new MIB can draw off the information from the old one. The only problem is, everyone's in a while we have to update our index of MIBs, because when a new MIB ob so let's an old MIB, you want to draw off on the new one. So the example I'm choosing is previously published so we run this S C M I C N G. And I would have extracted the MIB from the document and this is the file I want to check. So this is the file that's been created. Actually, sorry. Let me show you this first. This is the file that we're actually checking. This is the extracted MIB. This is the MIB file only. Future MIBs will have big boiler plate text. And we do another step to create a smaller file, and it directs, sort of directs the file, like what files to look at for this document. So, it's saying like include these files. Like we need to draw these files, or to check this. And finally, we're going to actually check it. BOB BRADEN: What's the source of these programs you're using here? SANDY GINOZA: Burt provided us with the extractor. I'm not sure actually. Historically, maybe Dave Perkins, we had to buy the book and get the initial checker, is what I think it was. BOB BRADEN: Okay. SANDY GINOZA: So I'm going to check this file and it's telling me there's an error. So this is an error that because we do this, we've seen this a lot of types, it's actually, according to what I understand, not a very big deal. XXXX suppress this error, and now it is success fill. So this would indicate where I would send the authors e-mail saying it parsed successfully, but we had to add these options to do it. Is this correct. Auth 48, there is no re demo. It's all author interactions rs, tracking e-mails. Everything you've seen before. So, the next thing we want to demo is the actual publication message. So, again, we would have gone into this find documents, the RFC is now assigned a number. So XXXX go to it. And I'm going to update this document with this entry. With all other relevant information. So I might do, if the title has changed, I have to update it. Any author addresses have changed. I'm going to include the date of publication. Once I think this, or include the abstract. Basically just updating this data page. Once that's been done, IAOC now go to create an announcement. So, again, I want to find the entry. And now create the announcement. So the announcement goes out to IETF announce and RFC editor and it cc's the working group that produced the document. And this is the standard RFC announcement that gets dristributed. So, I would, before I send this message out though, I would run the scripts to see that this document actually shows up in the correct place, it actually is available and then we would send this announcement. And the sending of this announcement would delete the document from the queue. BOB BRADEN: Then you start the next one. SANDY GINOZA: Right, right. SANDY GINOZA: So I'm going to kill this one too so I don't send this out. RAY PELLETIER: I could do one a week. Maybe. SANDY GINOZA: So, there's a few more slides. ALICE HAGENS: Just do the overview. [slide 59] SANDY GINOZA: This is just a brief overview of the exceptions cases. Like we said, so many documents come in with exception itself. It's not a straightforward submission to publication process. The most common things we see are changes that we think maybe beyond editorial, so we're going to interact the a area directors, and those can occur at any time during the process. Most of these other ones, there is already a plan, an action that we can take, when these things occur, so they shouldn't be significant halting points, but possibly. RAY PELLETIER: These are basically manual things that have to be resolved? SANDY GINOZA: Right, right. Obviously, I'm sure you guys know, based on the community, highly e-mail based. So, the blue boxes are the minimum number of e-mails that will be generated for any document. There's two potential places where we would ask questions, it would be during editor auth 48 and obviously reminders and that kind of stuff. So there's a lot of e-mails that are taking place. So this one is, if you can imagine, if you have the ID, you feed it to the RFC editor, you want them to publish this RFC, these are all of the individuals that come into play during that stage, so you want to track e-mails from all of these components, and it gets a little difficult, because you're tracking all of these for one document and you're tracking them for multiple documents in various states. So, just trying to make sure you're organized with all of that. [slide 62] And I think you guys know about RFC errata, but the only record of post publication fixes is via the errata pages. So you can submit on line, people can reporter Rohr's that they find and those errors will be sent to area directors, and the working groups, and the IESG or the stream specific parties, so, depending on the stream that the document originated from, that body has the ability to change the state, like they review the errata and decide if it fits into one of these four categories in blue, which scb defined by the IESG. And the search engine, after it's been, well, actually it shows all of them. But the RFC editor search engine has hyper links to these noting the errata for the document. This is the RFC editor role. Operate the web portal. Monitor the errata submissions and verifications, that becomes important, it's a rare case, but for example, recently somebody submitted an errata for the document on HTML pages, we don't have control over that. So we deleted that and sent the e-mail to hen Rick and said you you may want to update your files. Act as S S P for independent submissions, send period yod dictionary mine ders. Track the community requests for S S P in particular. Because if there's a way to make it more efficient, we need to do that. And the last one, is kind of hopefully, best case scenario, this doesn't exist, but pretty much errata for errata. PAUL HOFFMAN: It does exist. You got one last month. SANDY GINOZA: So if the errata exists, and the author says it's not cekts, if it's designated to any of those four categories, the S S P needs to request that we give them temporary write access to that file or make the changes. BOB BRADEN: So in a new regime this would be done by the RAY PELLETIER: Publisher would have the portal. All the other actors will play the roles. ( Same roles. BOB BRADEN: So who will monitor errata submissions and verifications, that's the production house? RAY PELLETIER: So, the way it happens now is that, somebody submits an errata, and there's an automatic e-mail that goes out. BOB BRADEN: Right. RAY PELLETIER: So, the publisher can still do that. Okay. rb on the automatic parts. RAY PELLETIER: Right. BOB BRADEN: My question is who does the send periodic reminders, I suppose that can be automatic. RAY PELLETIER: Right. Hound programming. SANDY GINOZA: So this is just some other basic information on some of the stuff that we do. S O W related statistics, more numbers based. Queue statistics, publication statistics, analysis of misref how many documents are there, how long they've been there. Errata statistics. For those types of numbers, you can just go to that link, additional reports that we do, would be to the IAB reports on a monthly basis, on the phone calls. To the IESG, per IETF meeting. And then this is just some additional stuff that we do. So, on tele chats, as a liaison, I would participate in the IESG phone calls, the IAB phone calls and the IETF management phone calls. We track these mailing lists so the IAB list and the IESG lists, see if there's any relevant issues, anyway we can be of help chiming in with questions they may have. Issues that are coming up. Additional lists that we currently participate in, I don't think that they're required to be participated in, Alice tracks the E D U team, XML two RFC and tools discuss team, changes that are going on those things. And E D U teams are NEW SPEAKER: What is the IETF management team meeting. SANDY GINOZA: We have monthly chairs with the IAB and IESG chair and Ray. NEW SPEAKER: Okay. ) Monthly phone calls. BOB BRADEN: Sandy, are we almost done, because we have to open the floor for questions. SANDY GINOZA: We are. And obviously, face to face IETF meetings. BOB BRADEN: I'll skip that unless people have specific questions. RAY PELLETIER: independent submissions, is there interest? If not, it is, the material is tr, the material is posted and people can do that and you can respond to questions. Okay. SANDY GINOZA: Questions and answers. RAY PELLETIER: And I think I had a slide for questions. Should we take a 5 minute SANDY GINOZA: That's a lot of information to digest. Sure. RAY PELLETIER: Yes. And lots of other things to digest. So I think 5 minute bio break would be good. (Recess.) RAY PELLETIER: Let's get started. This is the part that should take about half the time. But hopefully, all of what's gone on before has provided the answers to the questions that we've got up here. What questions do I have here? I had indicated to the world that, if questions were submitted in advance, that the source of those questions would be remain, anonymous. And so, I have taken those questions and put them up here, as kind of the beginning of what the Q and A period might be. So, some of these may have already been answered. There's 18 of these already. So why don't we see if we can't go through them and the expectation is that amongst the ISI folks and perhaps myself, if I have to, we'll jump in and answer these questions as best we can again, the official answers are to be posted. Right. Q1 The first question, how many different people touch each document along the way? And Sandy, Alice? SANDY GINOZA: That number varies, depending on who is working on it. It's possible the most could be three, there might be a separate copy editor and primary editor and myself or Alice would go through at the last stage and go through the RFC or it could be Alice or I who do the entire process ourselves and sebd it out. BOB BRADEN: One to three. RAY PELLETIER: I think one thing to note there is that, the RFC editor state, as described here, is intended to be a production center state. It doesn't mean the RFC series editor per se. That's a different person overlooking the series, right. Just consider that a state, don't consider that a person. Okay. Q2. How many staff are currently employed in the production center and woork full time, part time, seasonal. And I think you gave some full time equivalents. 4.75, plus Kate, it might be .25. BOB BRADEN: There's the programmer though, part of production center. RAY PELLETIER: And the programmer was part of that 4.75. BOB BRADEN: Yes RAY PELLETIER: It's in the neighborhood between four and a quarter and 5. So, that would be it. NEW SPEAKER: Is that for all four pieces? RAY PELLETIER: That is the, that does incorporate all four pieces in the sense that for example, Bob functions as the independent submission editor, as well as all of that -- SANDY GINOZA: I would say this, what throws that number off a little bit is people who devote their time other than what could be considered full time person. As independent submissions editor, I think Bob devotes significantly more than 25 percent of his time to tracking those things. BOB BRADEN: Which is why the queue is so long. Should. RAY PELLETIER: Right. So the point two 5, well, so, some number less than point two 5 might be production center kind of focused. I'm not sure, are you involved had the day to day editing. BOB BRADEN: No. RAY PELLETIER: Okay. So, I think that, so, I think, the number between four and a quarter and 5 or four and a half might be the right number. BOB BRADEN: Not to say when we were in trouble, in 2003, I did edit some documents. But since then, I have, these ladies have gone well beyond my knowledge. I couldn't do it today. RAY PELLETIER: The backlog, when that big peak in 2006, that had to do with the backlog. BOB BRADEN: That's right. 2006. RAY PELLETIER: And we hired a part-time outsider as well. So, seasonal, I'm not sure there's any such thing as seasonal, to address the burstiness or anything. First in or first out. BOB BRADEN: There is this phenomenon of the change over in April. RAY PELLETIER: Right. BOB BRADEN: Which causes a burst. RAY PELLETIER: However, do you hire any additional staff part time or anything to address that? Or -- we use what we have BOB BRADEN: We haven't needed to recently. RAY PELLETIER: The processing times may take longer to adjust for the additional level of work? BOB BRADEN: It's really, up to IESG what the IETF. RAY PELLETIER: q3. What tools are used and I think there's been a description of all the tools that are used. There's also a list that was provided of all the tools that are used. It's in the procedures manual, I think. BOB BRADEN: Yes. It doesn't, does not include probably the web front end or the my SQL database. RAY PELLETIER: It's also in the production center RF P. It's in that statement of work. There's a list of tools as well. NEW SPEAKER: Are there any tools that are managed or updated by like the tools team BOB BRADEN: No. RAY PELLETIER: Well, I think you just heard the XML RFC is an externally developed or maintained tool. And RFCdiff ALICE HAGENS: The ABNF parser is available from the tool site. It was made by Bill fin ner but originally had (Several people talking.) He had it on his own. NEW SPEAKER: ALICE HAGENS: They put it on the tool site to help us out. Because it used to be unavailable. >> RAY PELLETIER: XML, or, there are there are tools that are under consideration now for movement from let's say volunteer tool status to let's say secretariat status, in the sense that they will move to the secretariat site, and they'll, they may still be maintained by volunteer, but at some point in time, the paid programmer might wind up taking over maintenance of those tools. So there's going to be a big discussion about this migration. Q3b Are all these IETF owned? RAY PELLETIER: The short answer is yes. BOB BRADEN: The contract says you own, it's not the IETF, it's the Internet Society. RAY PELLETIER: Right. It's work for hire, all of the programming was paid for, and Jon and had this conversation, we have a meeting of the minds, we understand that. So the short answer is yes. The others part are the volunteer tools, but, you know, obviously, the license on that has permitted them m the past to done that. And we have no expectation of anything different happening there. We can continue to use those tools. PAUL HOFFMAN: Before you go on, Ray, that's an important question. You mentioned the front end to my SQL database, is that a tool that's IETF owned. RAY PELLETIER: The short answer is yes. BOB BRADEN: Yes. PAUL HOFFMAN: I'll look at you when I ask the question. (Several people talking.) BOB BRADEN: When Ray says tool I interpret that to mean software. Support software. PAUL HOFFMAN: Okay. Good. RAY PELLETIER: But we'll take their Macs too. We'll work that out. (Several people talking.) PAUL HOFFMAN: Q3d. And the individual scripts and such fall under that. BOB BRADEN: There are probably 80 or more scripts. RAY PELLETIER: Yes. PAUL HOFFMAN: The front end was the bigger one. RAY PELLETIER: How much time is devoted to interaction with outside agencies like IANA for editing documents. Sandy and Alice has gone over the process, time wise per document basis. BOB BRADEN: 20, 30 percent? RAY PELLETIER: If a document requires registry work, and 20 percent of the documents do or some number, some percentage? BOB BRADEN: You have to include the time you spent on the phone listening to IESG chatter. RAY PELLETIER: Well, it says for internet documents. Well, so, ten percent of the documents have IANA interest? SANDY GINOZA: Significantly more than that. RAY PELLETIER: 40 percent? SANDY GINOZA: I would say over 50. I don't know the exact percentage, but definitely over 50. RAY PELLETIER: And each of them require an hour, is some of that just e-mail or some of that actually on the phone? SANDY GINOZA: It depends, rarely is it on the phone. If there's questions I don't understand something or I don't know where, how they assign these values or why they don't match. I might call Michelle at IANA and ask her questions. If they're simple, there are hardly any time at all. It might be the assignment of a MIB value and we insert it and it's done. One minute. Some documents are highly IANA intense where they assign, they create new registries and multiple registries and sub registries and they all populate them with values and so trying to figure out which values go where and if the registry matches up, that's more difficult. I can't actually give you an exact time value for each document. ALICE HAGENS: But even for the minimum case, like where there's one value that's inserted, you're still doing the bear minimum of flagging it as IANA, awaiting the mail, changing the state when the IANA actions have been completed, changing the state and notifying IANA so, the bear minimum is 40 minutes or SANDY GINOZA: Ten minutes for a simple case. RAY PELLETIER: Okay. Susan, are you still on the call. NEW SPEAKER: Yes, but I haven't heard really anything. I've I sent Ray e-mails about that. RAY PELLETIER: I just now see the emails at the very top of my list. Again, the transcript of this will be made available. And, Meghan, are you on the call too, or John? NEW SPEAKER: I'm still here. RAY PELLETIER: Good. Why don't we move on. And Susan, we'll make sure that the transcript is made available. NEW SPEAKER: It would help to see some form of demo, because there were clearly things that I couldn't hear and I don't know how to do that, and it would be interesting to see. RAY PELLETIER: There's no video recording of that, so that's -- NEW SPEAKER: Screen shots or keys? It sounds like you were doing something online. RAY PELLETIER: The only thing that, the only screen shots that I think had some value to it might be the screen shots of the front end of the database, where it showed the fields to be, data to be collected. RAY PELLETIER: Specifically, in No. 5, specifically regards to IANA what has ISI established for communication methods to insure timely processing in regards to registering and inserting necessary protocol parameters. BOB BRADEN: I think we covered that in detail? RAY PELLETIER: Yes. That would be the e-mails, principally, BOB BRADEN: The irony of course is that IANA is on the third floor of this building, but we don't ever take advantage of that fact. RAY PELLETIER: No. 6, was the, was the edit pre approval process that you tested considered successful? Would you recommend pursuing this option moving forward? This was kind of the field editing. There was a test, and Alice was involved in that and reported on that in the IETF plenary and other places. ALICE HAGENS: Right. There were six documents and they were all XML source files. Basically, small sample size, so hard to call it a success or a failure. But, about half I think had significant changes by the time they came to us, actually approved for publication. So they tried to give it to us for a pre edit can. With the idea that when it came to us for real publication, it would be in better shape, because we had seen it before. Because we'd already fixed things. But the reality of it was that the documents changed a lot. There were, you know, 5 versions or three, you know, over three versions that had gone through before it came back to us. SANDY GINOZA: I want to say some of them took over two years to get published before they were actually ready to be into our queue so significant time lag where you're not as familiar with that document anymore. ALICE HAGENS: Right. RAY PELLETIER: Was the field edit done at the work group last call or was it much earlier than that? ALICE HAGENS: I can't remember. They solicited volunteers to, who wanted to have their draft have pre edit, but I'm not sure they made a requirement to make it be a certain requirement at a process. PAUL HOFFMAN: They didn't. One of mine was one of the ones thrown in, it was close to last call but not in last call at this point. RAY PELLETIER: Would you recommend pursuing this option moving forward? Any thoughts on that? BOB BRADEN: My feeling was it would be significantly more expensive. Because, batch processing is the most efficient way to keep a resource busy. Like an editor. RAY PELLETIER: This is a topic that crops up everyonce in a while on the list, and right now it's active on the list, the question about doing, editing, what I would call field editing and where that field editing might take place and under what circumstances. A lot of the discussion now seemed to focus on English as a second language where we have documents in that particular state that perhaps could generate some real value to do some editing at an earlier stage such that other people can then have access to it and understand it better, and perhaps progress the document faster. But again, it seems to be a community discussion BOB BRADEN: Another reason it would be less efficient is that the editor would end up going through the same document N times. And rather than once or twice. RAY PELLETIER: Right. BOB BRADEN: It's a luxury that maybe you you can't afford. RAY PELLETIER: Of note, it's an optional possibility for the production center in the RFP and it won't be triggerd until such time as decisions are made at a higher level, and at that point in time, could be soliciting proposals for the cost. So, No. 7, what's the average volume of follow up required for each document? And I'm assuming it's follow up with let's say authors SANDY GINOZA: Again, that's dependent upon the document. The document size, how many authors there are. It varies significantly. BOB BRADEN: But what's the average? SANDY GINOZA: I couldn't even begin to tell an average. Alice, do you have a different answer? ALICE HAGENS: Are we talking specifically about asking author questions? BOB BRADEN: Yes. How much time do you spend doing that? ALICE HAGENS: I'd say, the mention earlier, the process evolved to ask more questions, and ask more questions earlier, so that there, you know, everybody is on the same page by the time it gets to auth 48 means that it's a rare case that we have a document go through the entire process without sending any. Or maybe I shouldn't say rare, but, it's less frequent that we have documents that there is no follow up or no author queries. But, average, three, BOB BRADEN: I think I see an average of once a day go by. At least. SANDY GINOZA: It's to everybody's advantage to ask questions. RAY PELLETIER: Number 8. ISI has been the RFC editor since 1977 why are they choosing not to bid? BOB BRADEN: Well, we've, so, it's a complicated answer. We've been doing this for ten years since Jon died, and it just seems like a, enough time in the pits. But, ISI is a research institute. We have 300 employees, which some are, over a hundred are Ph.Ds, and in computer science. And our mission is research. The internet was once the subject of research. We were paid to do that. But the government no longer funds internet research and so we're moving onto other things. So there's not the infrastructure. The 11th floor here is the networking division of ISI. Jon had his office and so on. And I had my office, have my office. And where the RFC editor lives. But, there's no institutional interest in continuing. RAY PELLETIER: Susan, can you still hear? NEW SPEAKER: I think it's in your equipment. Go ahead. RAY PELLETIER: We'll try to speak up louder then. NEW SPEAKER: I don't think that's what it is. It's not picking it up, (inaudible) I don't think speaking louder would help. RAY PELLETIER: Okay. BOB BRADEN: I guess the other comment I would make is that we are very aware that transition will happen some day. And you guys have got to work out how to make it work. And why delay the agony? RAY PELLETIER: Okay. No. 9. What's the most time consuming part of the production center work? How would you allocate the time? SANDY GINOZA: The most time consuming part should be the initial edit stage, right, so the first time we're reading the document entirely, going through it, making the edits, interacting with authors, getting it ready for publication. I think is typically the most time consuming. Do you guys have any -- RAY PELLETIER: And, the question is looking for some percentages, one might spend in one's day. Having said that, would you think that edit state for example, you spend 75 percent of your time in editing work and the balance of your time doing the RFC editor state auth 48 and other administrative liaison kind of work? BOB BRADEN: I would say 25 percent on meetings, liaison, is probably reasonable estimate. RAY PELLETIER: Is that across the staff? BOB BRADEN: Yes. Going to IETF meetings and liaison, and working with IANA and answering the other questions. RAY PELLETIER: Ten, what is the most challenging part of the production center work and why? ALICE HAGENS: I was going to say consistency. A series that's 40 years old and same thing that we're talking about during edit state, you can get caught up trying to be consistent not just within the document or set of documents but across the series. BOB BRADEN: I would say the hardest part is the judgment necessary to decide what to fix. How far to go in being professional editors, and constant pressure of work. We have to compromise. And, that's tough. Decisions. RAY PELLETIER: Anything else? RAY PELLETIER: So ISI has made a lot of improvements over the years, what additional improvements would you consider still critical for a more efficient process? BOB BRADEN: One of the tasks that we talked to you about but has never reached the top of the list in priority is a portal to keep track of the database to keep track of auth 48. But you know, it's, it would be useful but it isn't clear that it's an urgent problem. Whereas the clustering for example, was really important to do. RAY PELLETIER: So, refresh my recollection, auth 48 wasn't part of this last programming -- BOB BRADEN: That's when the authors keeping track of the multiple authors of each document, whether they have approved it, RAY PELLETIER: Right. The question had, my question had to do with, the last programming funding that was provided, did not include the auth 48? BOB BRADEN: I don't remember. SANDY GINOZA: I think it was part of that, but I think we prioritized that lower with the headers and boiler plate document. Programming requirements. RAY PELLETIER: Okay. Is the time remaining available on the contract, or from your part time programmer available to get it done by the end of the year? BOB BRADEN: I'm sorry. That question is out of bounds for this meeting. RAY PELLETIER: We'll take that off line, and we'll -- PAUL HOFFMAN: Well, it is important for people bidding to know what tools, in addition to what we've already seen are already going to be done, so that they aren't bidding, like if someone said that was a critical tool for them, to know whether they need to budget that in for next year or whether that's going to appear between now and the end of the year. So, I'm fine with you guys having that discussion off line. BOB BRADEN: My recommendation to you would be don't worry about it. PAUL HOFFMAN: Meaning you don't expect it to be done? BOB BRADEN: I don't think any programming we do between now and the end of the year will make a significant impact on your budget. PAUL HOFFMAN: That's sort of (Several people talking.) BOB BRADEN: There may be some incrementalism. RAY PELLETIER: And my expectation with regard to a contract might have more to do with, let's say a cost per hour, rather than a project cost. So when we get around to doing some development work, we might define a project, get a proposal, and then work to see how we're going to fund it. PAUL HOFFMAN: It's not even clear that it would be the production center who would be the ones doing it. RAY PELLETIER: Exactly. Let's make sure we follow up on getting this -- BOB BRADEN: The other two areas that we've talked about are, obvious problems areas from the discussion here, are the keeping track of communications with IANA. In fact, making the communication interface to IANA more efficient. We met with them and tried to arrange, we improved things but it's still very complicated. Whether anything could be done in an automatic way to help IANA, I don't know. RAY PELLETIER: Right. That particular item is something that we have on our to do list. BOB BRADEN: And the other is, whether there's, whether we should somehow merge our information system with the secretariat's. So that the, our database somehow is coordinated with theirs. RAY PELLETIER: Right. And when you say secretariat's, you're talking about the ID tracker itself? BOB BRADEN: Yes. RAY PELLETIER: So that's the IETF tracker. SANDY GINOZA: I was confused. I guess my personal opinion about the IANA tracking, I'm okay with it. But I thought initially when you were talking about it, it was the coordinatting IANA interactions, the secretariat database, IANA and our database together, which I think was discussed a while ago. RAY PELLETIER: And is an objective. BOB BRADEN: It is an objective. RAY PELLETIER: And it will be done. It's only a matter of when it's done and it will be done. BOB BRADEN: Yes. RAY PELLETIER: And we understand that that is, we, the IETF, is engaged had the underwriting of getting that done. BOB BRADEN: IAOC say pretty clearly, it will not be fully accomplished by the end of our contract. Hopefully it will be started but certainly will not be finished. RAY PELLETIER: You're probably right. BOB BRADEN: And we also don't know to what extent the restructuring itself will introduce new communication problems which will need programming to help. RAY PELLETIER: You meant new communication opportunities. Okay. I think we've exhausted that particular topic. What unique skills are required for this job and why? I'm assuming this has to do with the production center part. SANDY GINOZA: So, if I were looking for an editor, I guess, I would say somebody who is able to easely roll with the punches, and not engage in combative language with the authors. And while we are looking for someone who has a mastery of the English language, not so much so that every detail has to be correct. Because that's never -- BOB BRADEN: They can live with a split infinitive or two. PAUL HOFFMAN: I thought you meant roll with the punches, when we're obnoxious assholes to you. RAY PELLETIER: Find another word for that. PAUL HOFFMAN: Something stronger. NEW SPEAKER: Dollar signs are always good. RAY PELLETIER: So somebody with diplomatic skills. SANDY GINOZA: Right. Somebody that can acquire the technical language or they already are familiar with it. BOB BRADEN: Fast learner is important. SANDY GINOZA: Also somebody is not so grammar oriented that they have to enforce every single grammar rule, because we often go against common -- RAY PELLETIER: I know in the four plus years that I've been here working with you, I know we've had a number of times that we've taken somebody from scratch and trained them to be technical editor, gone from copy editor to editor stage, what's the full time person, what's the learning curve, how long does that take? BOB BRADEN: Stacy, how long have you been be here? NEW SPEAKER: I've been here since August, and I don't know, I feel like I have made stride. SANDY GINOZA: Towards the end of the year. NEW SPEAKER: November, maybe, I felt, took a few months. Three or four. RAY PELLETIER: Three or four months. NEW SPEAKER: And this is coming from somebody with no computer background and purely English background so it took me longer than it might have taken somebody with computer. SANDY GINOZA: My guess would be, it's pretty similar. ALICE HAGENS: Currently we have Stacy and Meghan both have masters degree in English, basically, so they're, what Stacy refers to as lack of computer skills is not really -- SANDY GINOZA: They have skills, yes. RAY PELLETIER: But they're not into SIP. SANDY GINOZA: And the other thing is, it's a continual learning process, we learn new things every day we meet at these meetings and there's things in these documents that we've never seen before and how are we going to handle these things. There is no definitive, you're done. RAY PELLETIER: Plus when you say you're familiar within three or four months, you're talking about the process of the editing. NEW SPEAKER: Right. RAY PELLETIER: But the consistency, within the cluster of documents and then the style manual and then the 40 years of history, I mean, that's ongoing. NEW SPEAKER: Yes, yes. NEW SPEAKER: So with the 40 years of history though, you have somebody to refer to, so, whoever gets this, is there somebody, you know, once the whole thing has been handed over, is there going to be somebody that can be called? RAY PELLETIER: Is there a guru? SANDY GINOZA: That's -- (Several people talking.) BOB BRADEN: Series editor. RAY PELLETIER: That's what the RFC series editor is, is the master. NEW SPEAKER: But a liaison here. RAY PELLETIER: Yes, there's going to be regular communication. Although, that position is not expected to be a full-time job for somebody. It is anticipated through the style manual, because the responsible for the style manual, through that they impart a great deal of that, right, consistency. Having said that, I'm sure that on a regular basis, there will be dialogue between the production center and the editor, and that's nothing that, you know, I or the I A O C for example, as contractors are going to obliterate. It's something that we really want that communication to occur. And I think it's one of the concerns that those who looked at how the restructuring was going to happen, that was one of their concerns, whether the communication was going to be there, and we certainly expect and hope that that communication is going to develop in ways that are appropriate for that to happen. E-mails, phone calls, visits. That kind of thing. SANDY GINOZA: Could I clarify this, I actually understood her question differently. RAY PELLETIER: I hope so. SANDY GINOZA: I could be wrong. I thought she was asking, yes, there's the RFC series editor, but is there contact with someone currently on staff, for historical reference, I guess. NEW SPEAKER: Right. Because I've done jobs before where, you know, nobody has any historical reference at all and it's like, who do you ask. RAY PELLETIER: I think, you know, the IAOC is interested in receiving bids from number of people and hopefully, they'll have people on, within their proposal that could do that. 13, given the production center and publisher functions may not be provided by a single vendor, what type and how much interaction would you estimate would be required between the two. Likewise, what type and how much interaction would be required with the RFC editor and independent submissions editor. I think the first part, was answered with the different steps, they went to the publisher role and described the state of affairs of the document, for example, at that point in time. And what the statement of work says for example, is that the expectation is the publisher is going to announce the document, have the document up, have the archive, and all of that, and that the production center is going to provide the publisher with a red ee to publish document. That you know, everything is done, you know, the file is done, everything is done, and the all that stuff gets passed. And Sandy smiles, knowingly, that has a caveat to all that. SANDY GINOZA: Well, it's a rare case but it does happen frequently enough where I would consider it a potential issue, is let's say I was the production center I've edited this document and I say it's ready for publication. I send it to the publisher function, the publisher function has to be ready to publish almost instantaneously, because there's a time lag, there's time for somebody to say, this document is available, there's errors, is there time to fix it. The document is not officially published so does it go back to the production house for editing? Does it stay with the publisher and do they have to have the knowledge to edit in XML or do we just say, no changes, done. We consider it published, issue an errata. RAY PELLETIER: At the point where you're describing somebody has said there's errors in the document, is that because that individual has been tweaked, nudging by something that you have done and you said, we're about to publish, do you have anything else you you want to say? SANDY GINOZA: No. RAY PELLETIER: Or it's because somebody off the wall, let's say an Alfred, someplace, just to pick a name. Has come in and said, you know what, I think there's an error here. BOB BRADEN: That's when errata occurs. RAY PELLETIER: That's errata. Now, it's a different state perhaps if the IESG says, wait a second, we don't want you to publish that document yet because of X, because every stream manager has the ability to do that, right. BOB BRADEN: I'm sure that will happen sometimes. RAY PELLETIER: And that will happen sometimes. And if somehow that split second happens, you've sent it and the publisher publishes it, we're another errata state. SANDY GINOZA: It's just this point. I feel that's very unfortunate, right. I would rather see the document published correctly. If somebody has noticed it before the announcement has gone out, why not fix it before issuing the errata. RAY PELLETIER: And the amount of seconds that it takes you between the point that you've decided this is ready to publish and the time that you sent the e-mail announcement, is how many seconds? SANDY GINOZA: Pretty instantaneous. BOB BRADEN: The fact is the publisher function is actually mostly automated. Scripts just run. RAY PELLETIER: Exactly. So SANDY GINOZA: So what I'm saying, if I send the e-mail to the publisher function, if they don't publish it, announce the document, put it in the correct repositories, there's this lag time that's created, where it's sort of in no man's land and at what state is that in? RAY PELLETIER: It's in, it's a, it's about to be announced state. SANDY GINOZA: So that's post -- RAY PELLETIER: It could be automated. That whole thing could be automated. NEW SPEAKER: If the publisher is hosting the tools, the production could still do the submit, send the e-mails, send everything out, you know, it could still be the production function to do the announcement. It's just that it's getting hosted on the publisher site. PAUL HOFFMAN: That's not the way the statement works today, I don't believe. NEW SPEAKER: It's a suggestion. RAY PELLETIER: Again PAUL HOFFMAN: Yes. RAY PELLETIER: Again, you know, while there might be clear bright lines as, described in the statements of work, it could be that, that we're going to learn over the next, over the year, that we need to improve the communications such that those bright lines are more gray. NEW SPEAKER: I would, I'm trying to say something here or not. But I think by dividing this up, we created more state in the system. And there's going to be states that don't exist now. Because things that are largely automatic operations will now be done by a different set of people. And so I think your suggestions for trying to have a set your suggestions for trying to have a set of tools keeping it more automatic is good, but it's probably not written in the model that way. So there's going to be some tuning, I hope. PAUL HOFFMAN: It's not in the model that clearly, but it's clearly in the statement of works currently, where the gap that Sandy is talking about exists. BOB BRADEN: No reason why the publisher couldn't have access to the my SQL database. RAY PELLETIER: Yes. I think, you know, this is going to be made to work. And we'll get all the parties on the same page just as who is pushing buttons. NEW SPEAKER: Hopefully a good example is right now, people on the IESG, make state changes m the data tracker that you don't, you host it, you host the tools but you don't, secretariat doesn't do that work, so that kind of model is probably the right one. NEW SPEAKER: Right. RAY PELLETIER: So, how much interaction will be required with the RFC editor and the independent submissions editor, I think the only thing we have not addressed there is the bit about the independent submissions editor, and what I would say, they're a stream manager just like the other stream managers and the amount of interaction will be somewhat similar. Between the independent submissions editor and the production center, as the IESG and the production center. BOB BRADEN: Well, except that independent submissions, independent submitters tend to be more cowboys and want to update their documents at the last minute. Or want special treatment. RAY PELLETIER: So you as independent submissions editor, once you've given the go ahead to, for the production center to start editing all of that, it's up to them then to interface with the authors, to get the document to a state where it's published? BOB BRADEN: Yes. So it occurs to me that time out state, which is the time, which is the state when the RFC editor sends an independent submission to the IESG for their approval, would not be an RFC editor state. It would be an independent submissions editor state. PAUL HOFFMAN: I think so. That was the way I read the model document. After that point, I believe any technical edits that the cowboy wants, the IESG can push back on and say I don't want to take that back. RAY PELLETIER: 14, given that a lot of tools used by authors are hosted on the tools web site which is hosted by Henric, is any time spent working with Henric to improve -- my expectation is that, you're currently working with Bill fen ner or marshal rose to improve the XML to RFC tool, for example, from time to time. ALICE HAGENS: Right. RAY PELLETIER: And other tool managers, yes? ALICE HAGENS: Right. RAY PELLETIER: That answers the question. Is yes, is any time spent -- ALICE HAGENS: Right. Yes, time is spent. Since we use XML to RFC as such an integral part of the process, I sent a lot more feedback, than about the tools about pages, but I definitely interacted with Henric, for example, the ABNF parser and extractor that takes the ABNF out of a document. And that's on the tool page, but it currently only takes a file name for internet draft, so I requested that it take a file for up load, because then we can do it for edited RFC file. But, so, yes, I mean, you see things available to the tooltion page that are helpful, and if they could be tweaked to be more helpful, then I try to ask. BOB BRADEN: but be don't have any control over his priorities. ALICE HAGENS: Right. RAY PELLETIER: And I learned that I don't either. lk so another big one is that the HTML version of RFCs have links to errata at the top of the document. And so, in rablingting with Henric to make sure that those links are there when they should be there, to go to the new, when we transition to the new system, in November 2007, making sure that those links went to the new errata pages. Et cetera. RAY PELLETIER: I think that maybe that's a signal. (Blank projector.) RAY PELLETIER: Okay. What was the last page, didwe cover everything on the last page? NEW SPEAKER: Yes. RAY PELLETIER: Can you explain the burstiness of the submission of documents. We know that temporary junior editors have been hired in the past to cover, how would how was the solution worked, what would you do to improve this. We noticed that when area directors are rotating off. And there's a lot of burstiness there. There's some burstiness with the IETF meetings themselves. PAUL HOFFMAN: And low times around Christmas. BOB BRADEN: Yes. RAY PELLETIER: Yes. So, but, I think you described earlier, that you don't hire temps in order to cover this. And, BOB BRADEN: Well, if we were running much closer to saturation, then we would probably have to. RAY PELLETIER: And I know through our experience that it hasn't been so easy to hire temps to, either that's an area thing, or a, USC thing, or, SANDY GINOZA: I want to say, partially it's the training time. If we're saying it takes Stacy three months to get acclimated by the time we invested in training, we are not editing documents and puts us further in backlog and we haven't accomplished anything yet. RAY PELLETIER: Plus a temp is six month temp SANDY GINOZA: It could be any length of time. RAY PELLETIER: But not longer than six months. I thought there were restrictions. ALICE HAGENS: Right. RAY PELLETIER: What does a typical workday look like, can you walk us through the various tas; I think we've had the walk through the various tasks part. SANDY GINOZA: That's similar to a typical workday also. RAY PELLETIER: Except that there are more than one document. SANDY GINOZA: Right, right. ALICE HAGENS: We didn't show e-mail either. A lot of large part of your workload is monitoring e-mails that are regarding documents that you're working on. So for example, Stacy might have six documents in progress, at least, like, you know, four that are in auth 48 currently that she's waiting for replies on, one that she's actively editing and one that she sent questions to the author, edit state so it's sitting in auth state waiting for reply. So then there's monitoring six draft strings basically. RAY PELLETIER: Right. And that's part of what the programming thing is that we'd like to get, right? NEW SPEAKER: I have a question about the e-mail portion of that. Is that tracked in any e-mail, you know, when you send an e-mail to an author to clarify something and algorithm that exchange back and forth, you know, hopefully over six weeks but maybe six months, how is that tracked? Are they sending it to you personally? (Several people talking.) RAY PELLETIER: A ticket system. NEW SPEAKER: Exactly. SANDY GINOZA: We don't have a ticket system. We have an RFC mailbox. They are set up as user. If we send individual males, it has to be sent to the RFC editor. We don't have a ticketing system. BOB BRADEN: Archive. NEW SPEAKER: On so how does that work if you you are trying to go back or there's a question and you haven't gotten a response from an author for six weeks because they're on vacation, finally they get back to you and it's regarding a specific document, how do you track that? Or what if you're out on vacation and Sandy you have to fill in? I mean, how do you, it sounds like there's no automated place where you go back and see the thread. SANDY GINOZA: It's not automated, it's, it's easy for us to go through. So right now we can use Mac mail and do a search for our initials or the RFC number and it will bring up all related ee mace and we can see what happened most recently or check history to see what went on with that document. It's more of a search feature as opposed to having -- NEW SPEAKER: Not a thread. RAY PELLETIER: Do you have any reports for example, that would tell you you that, you know, certain time has gone by and nothing has happened to a document, and that it, demand some attention? SANDY GINOZA: So, the way -- RAY PELLETIER: A dashboard kind of thing. ALICE HAGENS: It's the weekly report that you see, that has time and state on it. We would say, why is this document Interstate 5 weeks. RAY PELLETIER: So in a similar fashion when we discuss this on a monthly basis and the IETF management so described meeting, that when we talk about the clusters and we talk about misrefs and all that kind of stuff, you are also looking at all those reports. They're flags to you. ALICE HAGENS: We're not so inltd in misref. You like to look at a different part. RAY PELLETIER: That's what I figured. Yes. Let's see now, I think 17, what does a typical document submission process look like, I think we did that. And walked through the various steps and we can go to question 18. Question 18 section G of the statement of work requires that at least one member of the production center needs to be at each IETF meeting, how would the IAOC prefer bidders to estimate travel costs for this. What I would tell you is that inputting together a budget, this may be one of the easiest, this may be one of the easiest parts. And that is, since we know what the meetings are for each year, is guestimateing what those travel expenses might be for that, I estimate my travel every year, and I estimate travel for other parts of the IETF process. In our budget. And, the good news is we do know where our meetings are, and just use round numbers. SANDY GINOZA: I would actually for that question be curious, it says at least one member, how do you know how many people you actually need to send? RAY PELLETIER: So, let me ask you a question, you've been doing this for a number of years. Essentially, the three of you come to each meeting. BOB BRADEN: Yes. RAY PELLETIER: And, you know, there's training going on, there's the help desk, going on, Bob is meeting with a lot of different people about a lot of different things, and I guess, most everybody leaves on Thursday morning or something. So, what is, what is your travel experience, it's arrive Saturday, leave Thursday? I'm not sure if that's the same for you, Bob. BOB BRADEN: I usually arrive Sunday and leave Thursday. SANDY GINOZA: I guess it depends where it's located. But yes, I would estimate Saturday to Thursday. But I would think, that's a little different for us, because these processes are going to be split amongst different entities now, so, maybe not everybody needs to be there for, or different representatives for different things. BOB BRADEN: It's hard to know. There's a lot of benefit, at least for the senior editors to meet their enemy face to face. And answer questions too. We staff a help desk and answer questions. RAY PELLETIER: And the community interaction process is with the authors and typically shs it seems to be a production center thing. It's not the IESG, because we can always capture the IESG folks. As opposed to let's say publisher function, right now, och the top of my head, I'm not sure -- BOB BRADEN: I don't see the publisher. RAY PELLETIER: I don't see the publisher sitting the help desk for example. Nor providing the tutorial part of that. SANDY GINOZA: BOB BRADEN: Series editor. RAY PELLETIER: I think the series editor ought to go. Definitely ought to go. SANDY GINOZA: Series editor. I think it also might be helpful for the I S E to sit down and offer some office hours time. ALICE HAGENS: Definitely. RAY PELLETIER: I can see the value of that as well. PAUL HOFFMAN: Sure, so many regular IETF ers don't understand the independent submission process. RAY PELLETIER: Right. So there could even be, there's a tutorial in there someplace. Let's say a Sunday tutorial for the I S E to talk about that and maybe solicit more. BOB BRADEN: You can always read our web site. RAY PELLETIER: It's not the same thing. It's not the same. That's why I like Skype. PAUL HOFFMAN: Ray, your answer to this question is, because they were asking how to, you know, one of the things was, how the IAOC do the travel, no, just do this on a year to year basis and figure out you and whatever task you're doing are going to need to do. RAY PELLETIER: Make a proposal and submit it as part of the budget. I can tell you there will be negotiations around the budget. And we ask, well, where did you come up with this number, that kind of thing. PAUL HOFFMAN: Yes. RAY PELLETIER: I think that's the questions that had been submitted previously. I'm sure that we just exhausted your patience in your seating capacity, or sitting capacity. Are there any other questions from folks, Mary, anything else you want to, before you glaze over entirely. NEW SPEAKER: No. I'm not glazed. No. That was a lot. Training period? Once you decide who? I mean? RAY PELLETIER: So the question is, is there an expectation on behalf of the IETF community and others that there's going to be a ramp up requirement, and that, instead of getting, you know, 25 to 30 documents every month, that that number might be lower than that in the beginning, and the short answer to that is, yes. You know, hitting ground running is difficult thing, whoever contractor comes in to replace anybody else. And so, there will be, you know, a period of time that we'd expect a ramp up period. So what's a reasonable period of time? Is it one quarter, two quarters? It's certainly not three quarters. We ought to have this thing under control within six months I would think, in terms of, you know, the communications, the coordination, and start to crank out 25 to 30, toward that, you know, between the 4th and 6th month, I would think. NEW SPEAKER: Sure. RAY PELLETIER: Your mileage may vary and your opinions may vary. Do you have some different thoughts on that? SANDY GINOZA: RAY PELLETIER: Depends who is doing it? SANDY GINOZA: I have nothing. It depends who is doing it. RAY PELLETIER: My expectation, if you're bidding on this and you think your competent enough to do it, our expectation would be that in the second quarter, you ought to be doing the editing part. You ought to be getting that down. So that, good question. Any other questions? BOB BRADEN: Maybe she should start in December. RAY PELLETIER: Well, of course, I mean, there is a transition period. We expect to have people under contract in October, and we expect that there will be some, you know, overlapping and -- BOB BRADEN: Just ready to help in any way we can. RAY PELLETIER: And that offer is much appreciated, and so, are, expect that there will be that training period. NEW SPEAKER: And would people be here or? RAY: Yes, here. Paul, anything else? PAUL HOFFMAN: The publisher, has that been decided? The statement of works says, it will be by this date, or such and such will happen. I haven't seen either. RAY PELLETIER: I'm going to cover that in the wrap up. And that's some job, we have to segue into. Do you have anything else you want to ask? PAUL HOFFMAN: No. RAY PELLETIER: Not to put you on the spot. NEW SPEAKER: We're good. RAY PELLETIER: Let me get the wrap up, and we'll be able to finish this within four hours. Okay. Deadlines, final answer, time line. Reiterate some stuff. If you have nor questions, if the world has more questions, they're all due by, you know, anywhere in the world, 5 June, so this Friday. We're going to publish the answers, and this is the site where the answers will be published. And whatever is published is what the official response is, not necessarily what you heard here today. But probably so. I don't expect we're going to change too much of this. Can't wait to see the transcript. Here the time line, on the RFC, the proposal for the RFC production center. Okay. The R P C time line. On 19 June change to the RFP itself, that will be published. So if we wanted to correct something, then that will be published and you need to take that into consideration the proposals that are due on 29 June. Okay. So that's the last deadline for us, to make any changes to the RFP. Any changes to the statement of work, for the production center. (19 June) Okay. I think that's a typical kind of thing. The R P C time line, continue tion, 29 June, proposals are due 29 June. There's a committee put together that's going to review all the proposals, conduct interviews, there will be some testing, we'll provide two or three internet drafts for folks, and this is similar to what happened last time. Internet drafts for people to edit and submit, and we'll be looking for, you know, copy editing as well as editorial, editing at a higher level, that some appreciation of understanding of the style manual, and the consistency thing. PAUL HOFFMAN: Ray, how will those tests be graded, given that the style manual actually currently, I think correctly gives ISI a fair amount of latitude on things. RAY PELLETIER: We expect that one would follow the style manual as one would do if one had the contract. And that we will have an experienced editor who has RFC editor editing experience to review theming. PAUL HOFFMAN: Okay. RAY PELLETIER: So all I can tell you is everybody will have the evaluation by the same person the same documents, and so, I think it's apples and apples. PAUL HOFFMAN: Just given that the style guide, like I say, gives correctly a fair amount of latitude, in fact, Sandy was saying, they would go through a document and then go, no, we don't really want to do as much. It seems to me, a test would be, you know, for, it's fine to ask, can you do this at all. But it seems like a test where somebody does better than somebody else, seems sort of odd for that. I mean, certainly, there will be some things that are definitive in the style guide that you want do do. But it sounded like the vast majority of stuff is more touchy feelly. RAY PELLETIER: You might note where you would consider, this is a touchy feelly thing and I might leave this in deference to the authors, blah, blah, blah, or, you know, exhibit some of your, exhibit judgment. Exhibit professional judgment. And you might put some notes in there, at the beginning or end that said, you know, this is what we did and why we did it. NEW SPEAKER: So, I was not on the IAOC when this was done the first time around. But what I her, and this may be not true, I think this is correct, the results of the test were actually fairly clear. So, it wasn't, they weren't small subtle PAUL HOFFMAN: I can't tell you you whether it's true, because we were never told. NEW SPEAKER: I'm just telling you what I heard. I don't know that officially either. But I think the experience last time was this was very useful in selecting. RAY PELLETIER: It was informative. It was informative. PAUL HOFFMAN: Okay. I guess that would sort of re double my concerns, since we were the ones who are tested the last time, and we were never told yes, you did great or no you did terrible or anything like that. And it was probably on a similar test where you had given us a couple of drafts in progress and we copy edited them and we did do those notes and we felt, certainly during the management of the copy editor looked at it be said, we've got no idea which way this one will go, probably the same way you guys have that at times, you know, you said that sometimes you you will get a paper edit and skip many of them. We didn't know whether to go heavy, or go light, you know and it's pretty much the same style guide, you guys haven't changed the style guide much in three years. RAY PELLETIER: Use your best judgment. PAUL HOFFMAN: We did then too. RAY PELLETIER: Exercise your professional judgment as if you were publishing this as an RFC. BOB BRADEN: A lot more input now than you did. A lot more level playing field. PAUL HOFFMAN: Maybe. Again, we didn't get the results so we don't know. RAY PELLETIER: On the time line, so this is what's going to happen between 29 June and 29 July. IAOC is going to select a vendor for negotiations. Terms will be worked out between then and 17 September and by fifteen October we'll have a contract executed and the transition begins at that point in time. So that's the expectation there. Publisher time line, we're in negotiations with AMS, the IETF secretariat for the function. That's a decision that was made by the IAOC, from all the input, including the RF I responses. If negotiations fail then we'll issue an IAOC will issue an RFP. And at the end of the day, what do we know, one January, the contract begins. So whatever period of time is required, will collapse or expand. PAUL HOFFMAN: And make wonderful music as it does it. RAY PELLETIER: I'm sure it will. So the R E S and I S E time line, note the asterisk in the upper corner. It says subject to change. This is something that's always in motion. But the expectation now is that on 8 June, on or about 8 June the IAB is going to call for nominations for those two positions. Nominations will close on a month later. And that 13 July the nominees will be announced. Those who are willing to serve for the purposes of getting community input for the, on these individuals. About a week, about ten days to 14 days for that process. July, August, interviews will be conducted. Could be on the phone, could be in person, could be some of that in person in Stockholm, for example. While we're there at IETF '75. 15 September the IAB is going to appoint an I S E, that's the target. Also, at that point the R S E appointment, subject to successful negotiations of an agreement with the IAOC. The difference between those two positions is the I S E is a volunteer position, as is the volunteers at the IESG and IRTF, and IAB. But there will be expenses, I mean, we anticipate we'll be providing expenses for the I S E, as well as possible assist stants, part time or full time assistant for the I S E. The IAB will also a point the R S E subject to negotiations again. If R S E is a compensated position, it's not expected to be full time position. It's understood there may be more time needed in the beginning, than once that person you know, reaches a plane and understands the job and has got, got his or or her arms around everything. And there may be some compensation as well during that transition period, to help cover expenses and perhaps fees. We expect to execute agreements around fifteen October, again, as with the others. Transition and definitely assuming position one January. So, everything is pointing to the one January date. And let's see, definitely want to thank Sandy and Alice for the professional work that they've done over the years and continue to do, and it's been terrific. We owe a lot to an awful lot of people, Steve Crocker for RFC one, without the RFC one, I suppose we wouldn't have RFC 5400. So here we are, and so, thank him and obviously, Jon for the 28 years, can you imagine anybody doing this for 28 years? Amazing, one person. And USC since 1976, BOB BRADEN: When Jon came here it was '77, I think. RAY PELLETIER: All the years and we appreciate that, that was a real commitment and it's meant a lot to the community and certainly for the internet. Joyce Reynolds for -- BOB BRADEN: She worked for Jon for many years. RAY PELLETIER: And Bob, definitely, thank you, you've been here since the '70s, my God. Thanks a lot. We appreciate it. That's a wrap. And appreciate you all coming here today, and Susan, bearing with us as well as you could over the phone. And we'll get the transcripts out here, within probably 48 hours, I would like to be able to get that in a position, but, it may take us then until the 12th to get that, so I don't want to commit to the 48 business. I'm expecting to get it from Renee within 48 hours. If there's nothing else. Thank you all, and thanks especially to our ISI hosts. Thank you. (End of meeting.) /Transcription Ends