wiki:Winter2014/Meetings/Retrospective
Last modified 5 years ago Last modified on 03/20/2014 11:28:16 AM

Winter2014 Retrospective Meeting 19 March 2014

Review of W14 DM 6-mo development cycle.

Participants

  • Steve P., AndyB, Greg, Jacek, Jim B. (initial discussion), K-T, Perry, Russell, Simon, Mario, Robyn, Jeff, Dick, Frossie, Josh Hoblitt

Status as of 19 March

Last bug fix implemented, testing now, prior to release.

  • Now some overlap in version numbers w/old stack. All external packages that are not updated have the same name. Must re-build with name appendix (+0 or +1? Mario reserves +0).
  • Some table files have IF-statements--should not be in committed versions.
  • Release tomorrow. Need to up-rev user docs, and prepare announcement. Mario to work with Dick.

What went right, wrong, could have been done better?

K-T:

  • W14 was even more unusual than predecessors, in that most folks were not working on the release. Middle-ware and QServ teams are an example. Made coordination somewhat less important w.r.t. weekly stand-ups. Less coordination going on. Going forward, we may still be in a learning curve.
  • K-T has a bunch of intended tasks yet to do. Middleware meetings went on 3x longer than intended. Tended to discuss issues, rather than status. Would like to see more impromptu messages among team members, more often.

Jacek:

  • JIRA comments may help a bit to document discussion.

Information exchange

Mario:

  • Why is mailing list not sufficient?

JB:

  • One is aware than some conversations are better live than in e-mail. But, we did a lot of chat meetings that were not seen by anyone else. This can cause other problems.

Perry:

Most questions went TO Jim, not FROM Jim (his tech expertise helpful/necessary).

Mario:

  • Why is chat better than list? Worry about spam?

Perry:

  • We were working on a task w/an excellent design doc, so little need to discuss that. Also, small group w/specialized tasks. E-mail has round-trip inefficiencies (must first clarify what question is being asked, then understand solutions, all of which take 1+ Q/response in e-mail).

Frossie:

  • Has anyone tried persistent IM solutions (which feature archiving, accessibility)?
    • Hip-chat may work for DM (integrated w/Atlassian).
    • G-mail worked pretty well.

Steve, Greg:

  • Meetings 2x per week.
  • IM: need to archive to be useful. Policy issues w/multiple G-mail accounts at UIUC?
  • Weekly stand-ups: Steve has lost touch with what others are doing. We do mix technical subjects with administrative messages on lsst-data list. Easy to miss important stuff. E.g., Up-rev of git was a surprise, and generated messages that were not expected. (Mario: actually, we went off-road on that. Sorry.)
  • This cycle was different, in that we did not produce data products, so (Steve) thought the finality was less marked.

Many did not attend weekly stand-ups. Why?

Steve:

  • Assumed it was only for management issues.

Mario:

  • We do have a limit of 10 connections. But this is not usually an obstacle. We could go to 15.

Greg:

  • Thought the meeting format effective. Kept meetings short. Flexible, in that if issues come up they can be discussed. E-mail to list generally works. We were a little under-weight in terms of release process. When products were produced, we focused more on process.

Mario:

  • This was really a re-factor release, so a little special.

Andy:

  • Attendance will be light if meeting is optional. Team leads report status, rather than discuss issues, so interest level is not high.

Perry:

  • Miss hearing where people were on tasks. Managers meetings not useful here.

Jacek:

  • Push off more status info to status reports. Maybe JIRA will help.

Jeff:

  • People were supposed to be doing that w/status reporting tool. [ Silent laughter.]

Mario:

  • Did people find Robyn's status reports useful?
    • All: Not so much. [ed: Dick thought Robyn's reports were fairly useful, and a good example for others to follow. Reminders of past issues & resolution, links to important docs (status, design, etc.), aggressive pinging of attendees to embellish her notes, etc. were all excellent.]

Jacek:

  • No forum to discuss items that are not directly part of release tasks. QServ development had issues that required input from other stakeholders (apps/pipelines), but there was no formal mechanism to solicit/demand input.

Mario:

  • Plan was to rely on ad-hoc meetings, with posted minutes.

Steve:

  • Had trouble editing DM calendar for Pegasus meeting. (This is a bug)

Mario:

  • These nuisance issues should not stop meetings from happening.
  • I'm asking because if we evolve to confluence or JIRA, it is essential that people read important content.

DESDM:

  • We are an all-volunteer force, so formal mechanisms and organization is a struggle and cannot be imposed. Use diverse communication mechanisms, which helps somewhat but tracking is a disaster.

Simon:

  • Found stand-ups to be useful to motivate specialized meetings.
    • Our part (camera) needed lots of requirements gathering. Thus more discussion necessary. Stand-ups provided a focal point for either reaching a decision or continuing discussion. Most other tasks were introverted, but Camera required more interaction with broader DM stakeholder base.

AndyB:

  • How would we record face-to-face interactions? Doesn't seem to be a good way to do this.

Simon:

  • Russell & I tried to address that by posting discussions.
  • Threshold: only report if substantial discussions; hard to quantify "substantial."

Frossie:

  • One cardinal Rule used elsewhere: everyone required to use IM to record interactions. This is draconian.

Dick:

  • Opined that he doesn't actually *want* to review live recordings of discussions. This is too detailed. Instead, would like to see some synthesis: What were the issues of substance, what points were considered, what conclusions were reached.

Jeff:

  • Second Dick's comment. Want to see summaries; some current reporting is too detailed for me (as DM Manager); part of my job is to synthesize information even further.
  • As Mario mentioned: we are exploring additional videoconferencing solutions (including recording). Good A/V features, one-button connect, deployed to every institution, etc.

K-T:

  • Appoint a scribe for each meeting. Mario: second, we do this for Coord meetings to good effect.

Steve:

  • I found that useful (for middleware). Having short, finite meetings is great compared to other projects.

Mario:

  • This worked well for coordination, and reviewing drafts of note excellent for confirming that decisions we made were in fact made, and were good ones.

### [break]

Was the split into small Teams effective

Mario:

  • The split into small Teams was new, expect to continue this into construction.
    • Recommended book: "Team Geek". How to make large groups work well together.
  • Format was small, self-organizing teams. How well did this work? Want first to hear from non-leads of teams, then leads.

Steve:

  • Most projects I've been on have been lone-wolf projects. Data challenges did involve small (2 person) teams.

Greg:

  • Middleware was more of a set of research projects, but not immediately connected to other teams. Investment in this area was important in this cycle, and will impact others later. Ex: wiki pages on how to run a production is a cross-team need.

Russell:

  • Very effective. Likes programming in small teams. Not clear it went as well with broader group. Outside criticism was not always helpful. Some comments from functional lead were pointed, but he was too busy to pay enough attention and contribute more meaningfully.
  • Stakeholders need to have & invest the time.
  • Would be nice if small groups could cross institutional boundaries once in awhile.

Mario:

  • How does this compare to Apps meetings?

Russell:

  • Apps meetings were initially useful, but not so much toward the end of last cycle.

Simon:

  • This new approach worked very well.
    • It was difficult to keep pace when other stakeholders are not engaged.
  • Weekly meetings were a good way to keep things on track. Smaller latency than before. Sometimes we thought we had sign-off, then we found we didn't, so a focus meeting (cross-institution) was planned, which introduced latency. Some points get lost when translated to e-mail.
  • Would like a (prescribed) way to reach decisions.

We assigned roles to specific people - was this effective

Mario:

  • We assigned roles to specific people, and roles come with responsibilities and timely input. Was there sufficient time? [Ed: apparently not always.]

Perry:

  • Liked new approach of small team effort. Big concern: lost track of butler and camera progress, and this will be a problem later. Not sure how to fix, but too busy to read status reports or attending status meetings.

Andy:

  • I was a lone-wolf, so very efficient. Much more fragmented production cycle; did not feel well informed of progress.
    • Mario: how did interaction w/AndyC go? He should have kept you in the loop.
    • AndyB: didn't happen; AndyC was always busy with other things.

Robyn:

  • I normally work alone. I liked having a team. Because I do a lot of (random) tasks, they need to be factored into task list for release cycle. Rate of progress was usually not what appeared in plan.

Dick:

  • Though individual teams may have worked more effectively together, understanding their progress was uneven. (This depended upon teams willingness/ability to post relevant discussions to wiki, lsst-data list, etc.)
    • Status meetings somewhat helpful, but Dick misses the technical forum. You would be surprised how much I (your documentarian) pick up from these conversations, and how it helps me follow up issues. But, I would not abandon the new organization; I need to adapt documentation task to the new reality.

Jacek:

  • Tried to push discussion out of big team meeting to smaller (2-3) people. This was more efficient.
    • Only have people at meetings that are required to be there.

K-T:

  • Even if not part of decision meeting, need to "overhear" things. K-T had more opportunities to interact with his team, answer questions, keep tasks moving forward. Likes this.

Mario:

  • Seems like we're more efficient, but more isolated.
    • If you are working on a deliverable that requires input from distracted stakeholders, this creates tension.

Jacek:

  • Possible solution: require that people document things. If not documented in Trac (or JIRA), then thoughts effectively don't exist.

K-T:

  • Well, who documents? then we need people to read documents and assimilate them.

Mario:

  • Part of my job to be reading these docs, but I don't necessarily have time.

Jacek:

  • We need to know where to look. Doesn't need to be official. Ex: data ingest, which touches on pipelines.
    • We need to reach out to stakeholders and document this. This may relieve pressure on meetings.

K-T:

  • [Missed this while gobbling pizza.]

Jacek:

  • Like a hackathon, need to record the issues in the docs

Dick:

  • Well, one of my gripes about the soon-to-be-retired wiki is that information goes stale unless people actively keep documents up to date (or retire them). A month after they're written, it's usually hard to tell what's official or not, what information is still correct, etc.

Jacek:

  • Our master docs we try to keep up to date

Mario:

  • Simon, didn't you do this for Camera?
    • Simon: Yes, we started with at wiki page, and evolved into a formal doc.

Perry:

  • We document everything in code. This has downsides, including the level of detail. Maybe some documentation belongs outside of code?
    • Dick: would like to work with you on that. Not clear what info belongs in code vs. external docs.

Frossie:

  • Need a convention for organization for content.

Simon:

  • Design doc for us was separate, but (Perry's) was incorporated (incompletely) in code.

How well did we do w.r.t. deadlines/delivered tasks

Simon:

  • Worked better, but hard to map estimates of work to calendar days, when working on multiple projects or when there are dependencies.

Jeff:

  • If plans are too detailed, planning does not match reality. Need to keep a more abstract plan, which allows flexibility for timing/phasing. Trajective planning is different than task planning.

Mario:

  • We aren't necessarily good at estimating difficulty of tasks, or ancillary commitments.

Jeff:

  • A couple of ways to address in mgmt: underload, and add big activities (Meetings, etc) explicitly into plan.

Frossie:

  • 6 months is a long cycle. After 3 months did anyone look at whether you were half way done?
    • Mario, Jeff: no, but we meant to.

Jeff:

  • An issue was that Jeff's job became feeding higher-level management, particularly for reviews. This is more than a full-time effort. Mario:
  • We feed the Gantt, but don't get output in the form of spreadsheet or chart. No person is looking at whether we are falling behind.

Robyn:

  • Burn-down charts in JIRA are much more useful than Gantt.

Dick:

  • There are a couple of points to consider going forward.
    • The current cycle did not involve many tasks that required action by more than one team; few dependencies external to teams. This will likely be different in future cycles.
    • Also, need to be sure our way of doing business scales when the size of the team doubles. We will need to invest time in bringing new members up to speed. Perhaps in the next cycle management can intentionally include planning tasks w/dependencies, to calibrate the team's abilities in this area.

What do you think of JIRA messages and other capabilities?

Russell:

  • Getting waaay too much spam. Need to tune system.

Simon:

  • Having a hard time to figure out when completed tasks need to land in calendar space.

Want to evolve to coding sprints of (2-4??) weeks.

Jeff:

  • If sprints are too short, packaging overhead is too great.

Frossie:

  • Right time frame depends on individual person/velocity/commitments.

Simon:

  • Need to identify tasks at finer grain than we're used to. This worked really well for ImSim.

Frossie:

  • sprint enables changing what we're doing mid-stream, as appropriate.
    • But need a purpose and a stakeholder to evaluate end products or functionality.
    • Someone has to notice if the sprint was missed.

K-T:

  • Well, maybe internal stakeholders would suffice in some cases.

Jeff:

  • Where we're behind is User Interface, but this is lagging the project. This will change when prototypes begin.

Acknowledgements and Caveats

A hearty shout-out to Dick Shaw for these detailed notes taken over the span of the 2.5 hour meeting.

These notes capture his impression of the highlights and include what he could transcribe in real-time. Feel free to edit these notes if your recollection identifies missing salient points raised at the meeting or if a point stated deviates from the intent of the speaker.

A retrospective of the retrospective meeting is also forthcoming from Mario.

The emboldening was an add-on by the editor of the twiki page. You may add or delete emboldening as desired.