I undertook several in the different roles of 3rd member, specialist and president. In every case found it really rewarding. It is true that the promotion system is not flawless and you can only score on what you see in the SJARs. We had an understanding between the board members that if a candidate took more than 10 minutes to read then so be it. They were given the time we thought was needed. Remember each profession is different and do use different language / abbreviations. The specialist is there to answer any questions you have. We did ask the professions advisor for clarification on one board.
We also declared if we knew a candidate on the grounds of openess. You'll quickly realise that the scoring each of you come up will be in the same ball park, at the end of the day though if you as an individual use the same scoring method for every person you can't go wrong.
Enjoy it, they are hard work so make sure you take regular breaks so you don't become word blind. Over the years people have started to make better use of the front page. I never read the scores of recommendations until after reading the narrative (just my way). Reading a narrative showed me the evidence of what a person has achieved to support promotion, if the evidence is there, the recommendation becomes pretty irreverent.