FOI Request LEX3029, Schedule of Released Documents [PDF 79KB] (pdf)
Download cached file | Download from AEC--- Page 1 --- Attachment A – LEX3029 Document reference Document 1 Document 2 Document 3 Document 4 Document 5 Document 6 Document 7 Document For Information: Independent test of senate counts [SEC=UNCLASSIFIED] Bowland’s recounts for various scenarios... [SEC=UNCLASSIFIED] Corrections to the Senate Counting algorithm Corrections to the AEC's counting code, and recommended amendments to the Electoral Act, for Senate Counting & Scrutiny Research team notes on the Conway Teague paper RE: Corrections to the Senate Counting algorithm [SEC=OFFICIAL] Senate Counting and Scrutiny Date 20/07/2016 02/08/2017 31/08/2021 31/08/2021 22/09/2021 19/10/2021 18/10/2021 Decision on Access Release in part S47F – personal privacy Release in part S47F – personal privacy Release in part S22 – Relevance Release in full Document withheld S47C – Deliberative matter Release in full Release in full
This document is an index of seven internal Australian Electoral Commission (AEC) documents and reports, dated between 2016 and 2021, detailing independent testing, identified issues, and subsequent responses regarding the AEC's Senate election counting algorithm and scrutiny processes.
Its relevance to the FOI request is direct and comprehensive:
* Early Independent Tests (2016-2017): Documents 1 ("Independent test of senate counts," 20/07/2016) and 2 ("Bowland’s recounts," 02/08/2017) directly correspond to the initial tests that reportedly found no issues replicating the AEC's 2013 WA Senate count, with partial release indicating their inclusion in the FOI response.
* Conway and Teague Report (2021): Document 4 ("Corrections to the AEC's counting code, and recommended amendments to the Electoral Act," 31/08/2021), released in full, is highly likely the central Conway and Teague report or a direct output that identified significant discrepancies, proposed corrections, and recommended legislative amendments. Document 3 ("Corrections to the Senate Counting algorithm," 31/08/2021), also dated concurrently, further relates to these findings.
* AEC Response and Internal Review (Post-2021 Report): Documents 5, 6, and 7 (September-October 2021) reflect the AEC's internal processing and response to the Conway and Teague report. Document 5 ("Research team notes on the Conway Teague paper," 22/09/2021) indicates internal analysis (withheld as deliberative matter). Documents 6 ("RE: Corrections to the Senate Counting algorithm," 19/10/2021) and 7 ("Senate Counting and Scrutiny," 18/10/2021), both released in full, represent the AEC's subsequent discussions and actions regarding the identified issues, proposed corrections, and broader scrutiny processes, addressing the "AEC subsequently responded" aspect of the request.
LEX3029 documents [ZIP 1.82MB] (zip)
Download cached ZIP | Download from AECZIP Contents
- Document 1 - For Information_ Independent test of senate counts [SEC=UNCLASSIFIED] (A1861023)_Redacted.pdf (pdf)
- Document 2 - Bowland's recounts for various scenarios... [SEC=UNCLASSIFIED] (A1861038)_Redacted.pdf (pdf)
- Document 3 - Corrections to the Senate Counting algorithm [SEC=OFFICIAL] (A1861055)_Redacted.pdf (pdf)
- Document 4 - Recommended Amendments Senate Counting And Scrutiny (A1860962).pdf (pdf)
- Document 6 - Corrections to the Senate Counting algorithm [SEC=OFFICIAL] (A1861209).pdf (pdf)
- Document 7 - 18 10 2021 - Dr Andrew Conway and Prof. Vanessa teague - Senate Counting and Scrutiny (A1860875).pdf (pdf)
Document 1 - For Information_ Independent test of senate counts [SEC=UNCLASSIFIED] (A1861023)_Redacted.pdf (pdf)
Download file--- Page 1 --- From: To: Subject: Date: Attachments: For Information: Independent test of senate counts [SEC=UNCLASSIFIED] Wednesday, 20 July 2016 2:52:00 PM image001.gif image002.gif UNCLASSIFIED As discussed a member of the WA Greens, Grahame Bowland, wrote python code to replicate the Senate count. He found no issue with the AEC results. It was a completely independent activity, using the legislation as a source, and comparing to the distribution of preferences (I think that is what he is referring to). His blog post on the matter is not available through the SOE, so I’ve replicated the text below. For reference, the software is still hosted on GitHub here: https://github.com/grahame/dividebatur Counting the West Australian Senate Election Posted on January 25, 2014 by Grahame Bowland tl;dr – check out angrygoats.net/wasenate/ to see the full details of the WA senate election distribution – precisely how the count was processed by the AEC for both the initial count (#14votes) and the recount, and to see what happens if the “lost” votes are added back in, assuming they were counted right the first time. grab the code and run the distribution yourself, it’s over on github. Western Australia went to the polls with the rest of the country on September 7 2013, but failed to clearly elect six senators. When the votes were tallied a crucial exclusion on a margin of 14 votes was the difference between the election of Scott Ludlam (Greens) and Wayne Dropulich (Sports Party), or Louise Pratt (ALP) and Zhenya Wang (Palmer). The ALP & Palmer won on this count, but no result was declared and a recount was conducted. After the initial count, several people asked me how the Australian Electoral Commission (AEC) processed the distribution of preferences, and whether there was a way to verify it. To my surprise the AEC software which takes all above-the-line and below-the-line votes and runs a Single Transferable Vote (STV) distribution is closed source and not available for download. I was unable to find any publicly available software to run the count. This made it extremely difficult to verify the correctness of AEC’s results. While the recount was going on I got to work on some software to process the distribution. This was greatly helped by a detailed run log from the AEC’s software I managed to obtain. This told me exactly how the AEC’s software ran the distribution, including totals of votes for each candidate at each count, and precisely when candidates were elected and excluded. STV is really a family of voting systems – some details vary between jurisdictions. The information I could find on the AEC website on how to process the distribution was adequate only for educational purposes, and did not go into the fine details. This lead to much late night study of the Commonwealth Electoral Act. A bill of Parliament is a pretty awful means to document an algorithm, but everything needed is in Part XVIII. The AEC publish all the data necessary to run a count on their Senate downloads page. It is simply a matter of taking the number of above-the-line (ticket) votes and counting these votes as that number of papers following the party’s ticket(s). They also publish the form of every below-the-line paper lodged. Add those papers into one collection and you have all the information necessary to run the distribution, and determine who the elected candidates are. In two long nights of work, I was able to get this done – exactly reproducing the AEC’s results in my software. The code was pretty rough and ready, but I had functional tests which checked my results matched the AEC’s. Being afraid of rounding errors I made sure to stick with rational number representations of transfer values, and was slightly hopeful this might turn up a minor error in the AEC’s software. Disappointingly, but reassuringly, I found no fault at all in the AEC’s software. The AEC published above-the-line vote counts as the recount progressed. This made me hopeful I’d be able to run a count myself once the recount was complete and know the result before the official distribution was run. Unfortunately they did not enter the changes to the below-the-line votes (ballots rescued from the informal pile) until the last day, and did not make the data available until a couple of hours after they ran the count and declared the result. Gratifyingly my software agreed with theirs, the election now tipping to Scott Ludlam and Wayne Dropulich, the same crucial exclusion now going 12 votes the other way. Of course, the AEC lost 1370 votes which were not included in the recount. The preferences of those votes as interpreted during the first count were published by the AEC, making it simple to add them back into the recount. This swings the crucial exclusion back to a margin of 1 vote, electing Louise Pratt and Zhenya Wang. It’s impossible to know if these votes would have stood as they did if scrutinised again, and votes may also have come back from --- Page 2 --- lost informal votes. The full senate vote distribution for these three scenarios can be viewed in detail. You can flick through the counts one-by-one, or the summary page for each scenario will link you to the count in which any given candidate was elected or excluded. My software outputs JSON data descr bing the distribution process, and the user interface is an angularjs application which presents that JSON data. If anyone is interested in analysis or visualisation of the full distribution (as opposed to a simplified distribution only including above-the-line votes) the JSON files are linked for download. I would love to see some great animations appear from this data! If you want to check out the code behind this, it’s over on github. I have named the project dividebatur (Latin for ‘distribution’) and it is open source under the Apache license. If you’re interested in the STV algorithm itself, it’s implemented in these 700 lines of Python code (hopefully quite readable). I learnt a bunch of surprising things implementing the distribution. A bundle of “x” papers distributed at a given transfer value translates into a “x” times the transfer value votes, rounded down. This means that occasionally votes will be “lost” to rounding. What I didn’t realise is that those votes can also come back – if those papers are distributed in an exclusion, they may break by next preference in a way that avoids the rounding issue. Subtle details like this (and split group voting tickets, exclusion tie resolution, …) were tricky to get right! In conclusion, we’ve now got some software which lets us as voters verify that the rather complex senate distribution is correct, and study hypothetical scenarios if we want to. It is easy to determine what would happen if a party changed their ticket vote in a particular way, or consider what would happen if a state-that-shall-not-be-named stuffed up their senate election. I hope you find it useful, and please send me feedback and/or patches! @angrygoat on twitter or grahame@angrygoats.net. Disclosure: I’m a member of The Greens (WA). I don’t believe anything in this post is partisan, and I am speaking for myself. My software is open source and can be verified at your leisure :-) | Senior Research Officer Strategic Research & Analysis Section | Roll Management Branch Australian Electoral Commission UNCLASSIFIED
The document is an internal AEC email from July 2016, forwarding a January 2014 blog post by Grahame Bowland, a WA Greens member. Bowland details his independent project to replicate the Australian Electoral Commission's (AEC) 2013 Western Australian Senate election count using open-source Python code (dividebatur
). His initiative was prompted by the AEC's closed-source software and the difficulty in independently verifying election outcomes. After obtaining AEC run logs and studying the Commonwealth Electoral Act, Bowland successfully developed software that "exactly reproduced the AEC’s results" for both the initial count and the recount. He explicitly states, "I found no fault at all in the AEC’s software." His blog post also discusses hypothetical scenarios, such as adding back "lost" votes, and notes subtle complexities of the Single Transferable Vote (STV) algorithm, including "exclusion tie resolution" and rounding.
This document is highly relevant to the FOI request as it provides direct evidence for the "early independent tests (2016-2017) found no issues replicating the AEC's 2013 WA Senate count." It specifically contradicts the later 2021 Conway and Teague report's findings of discrepancies by stating an earlier, detailed independent analysis found no faults with the AEC's methodology or algorithm as applied to the 2013 election. It highlights the differing conclusions reached by external scrutiny over time regarding the accuracy and implementation of the AEC's counting processes, particularly concerning aspects like tie-breaking and vote transfers, which were later flagged as problematic.
Document 2 - Bowland's recounts for various scenarios... [SEC=UNCLASSIFIED] (A1861038)_Redacted.pdf (pdf)
Download file--- Page 1 --- From: To: Subject: Date: Attachments: Hello all Strategic Research and Analysis - NO; Counting The Senate Project - NO; Parliamentary Engagement and Party Registration Bowland"s recounts for various scenarios... [SEC=UNCLASSIFIED] Wednesday, 2 August 2017 2:54:19 PM image001.gif image002.gif Grahame Bowland is a private citizen and Greens member who has devised his own Senate count software. He has recently been running scenarios to cover the various s.44 possibilities, which can be found here: https://sirgraha.me/senate2016/#/index Just figured I’d raise awareness of its existence. Regards, Sam. | Senior Research Officer Research & Analysis Section | Roll Management Branch Australian Electoral Commission
This document is an internal Australian Electoral Commission (AEC) email from August 2, 2017. It informs AEC staff about independently developed Senate count software by Grahame Bowland, a private citizen, which he used to run various scenarios, providing a link to his work. The email's purpose is to "raise awareness of its existence" within the AEC.
Relevance to FOI Request:
This document is relevant to the FOI request as it demonstrates the AEC's internal awareness of independent efforts to develop and test Senate election counting algorithms during the 2016-2017 period. This aligns with the FOI request's mention of "early independent tests (2016-2017)" that "found no issues replicating the AEC's 2013 WA Senate count." It provides context regarding the AEC's knowledge of external scrutiny and attempts at replicating their counting methodology prior to the 2021 Conway and Teague report, which subsequently identified significant discrepancies and issues with the AEC's algorithm and processes.
Document 3 - Corrections to the Senate Counting algorithm [SEC=OFFICIAL] (A1861055)_Redacted.pdf (pdf)
Download file--- Page 1 --- From: Vanessa Teague <Vanessa.Teague@anu.edu.au> Sent: Tuesday, 31 August 2021 3:40 PM To: Tom Rogers Cc: andrewstvcount@greatcactus.org Subject: Corrections to the Senate Counting algorithm ; Thomas Ryan CAUTION: This email originated from outside of the Australian Federal Government. Do not click links or open attachments unless you recognise the sender and know the content is safe. Dear Mr Rogers and Mr Ryan, Please find attached our report on the AEC's Senate count. We have found one instance in which the software breaks ties in a way that is not what is specified in the Electoral Act. We have also noted some other ambiguities that need to be resolved either by amending the Act or by otherwise committing to the exact counting algorithm. We would be very happy to help you correct the code - you can see our implementation here https://github.com/AndrewConway/ConcreteSTV. We could easily make the necessary adjustments to the official code if it was similarly published online. Also, we support recent proposals to audit randomly selected paper ballots at the end of the digitizing process to give everyone evidence of the accuracy of the process. We would be happy to help the AEC implement these measures securely and transparently. Yours Sincerely, Vanessa Teague and Andrew Conway. Vanessa Teague CEO, Thinking Cybersecurity Pty. Ltd. --- Page 2 --- A/Prof (Adj.), Research School of Computer Science The Australian National University | Canberra ACT 2601 E: vanessa.teague@anu.edu.au W: https://researchers.anu.edu.au/researchers/teague-v CRICOS Provider #00120C
The document is an email from Dr. Vanessa Teague (co-author of the Conway and Teague 2021 report) to Tom Rogers and Thomas Ryan of the Australian Electoral Commission (AEC), dated 31 August 2021. The email serves as a cover letter for the attached report on the AEC's Senate count, which identified discrepancies in the counting algorithm.
Summary of Document Content:
The email highlights two primary issues found in the AEC's Senate counting software:
* One instance where the software's tie-breaking rule is inconsistent with the Electoral Act.
* Other ambiguities in the Act or the specific counting algorithm that require clarification.
Dr. Teague offers to assist the AEC in correcting the code, suggesting that publishing the official code online would facilitate such adjustments. The email also expresses support for proposals to audit randomly selected paper ballots post-digitization to provide evidence of accuracy and offers help in implementing these measures securely and transparently.
Relevance to FOI Request:
This document is highly relevant as it is the direct communication accompanying the "subsequent 2021 report by Conway and Teague" specifically mentioned in the FOI request overview. It confirms the nature of the "significant discrepancies" detailed in that report, namely "tie-breaking rules inconsistent with the Electoral Act" and "ambiguities in the Electoral Act concerning the count's termination." The email also directly conveys "various legislative and procedural recommendations," including publishing the AEC's code and enhancing audit processes, which were central to the report's findings and the AEC's subsequent response, thereby directly addressing the FOI request's scope regarding the accuracy, methodology, and identified issues with the AEC's algorithm and scrutiny processes.
Document 4 - Recommended Amendments Senate Counting And Scrutiny (A1860962).pdf (pdf)
Download file--- Page 1 --- Corrections to the AEC’s counting code, and recommended amendments to the Electoral Act, for Senate Counting & Scrutiny Dr Andrew Conway ∗ A/Prof Vanessa Teague† Thinking Cybersecurity Pty. Ltd. & Australian National University vanessa.teague@anu.edu.au August 31, 2021 Abstract This report summarises our recommendations for technical changes to those parts of the Electoral Act that regulate the Senate Counting and Scrutiny. We show, and explain how to correct, a point on which the AEC’s current counting software does not correspond to the requirements of the legislation. As far as we know, this has so far not caused the wrong people to be elected. We also identify ambiguities in the legislation, which might leave the result undefined. These would be better resolved before the election when they are politically neutral. Although the digitization process is separate from the counting stage, our results bolster the argument for an independent audit of the paper ballots every Senate election, to demonstrate to scrutineers that the ballot-digitization process is accurate, or to allow for any errors to be identified and corrected. 1 Introduction The Australian Senate count has several distinct phases: • first preferences are manually counted, • paper ballots are imaged at a counting centre, • the images are digitized, using a combined human and automated process, • the digitized preferences are electronically counted, in order to establish who won a seat. This report focuses on the last step. We find an example in which the AEC’s official counting software deviates from the requirements of the Electoral Act, and two examples of changes of interpretation. We then provide simple examples to demonstrate that these could, in principle, alter election results, though it does not seem to have done so yet. We then detail where either the code needs to be corrected to conform to the Electoral Act, or where the Electoral Act could be amended to match what we know of the code. The error concerns the resolution of 3-way ties for the candidate with the lowest tally. We find an example from the 2016 WA Senate election, in which the AEC’s software applied the wrong tiebreaking rule, resulting in candidates being excluded in the wrong order. This is described in Section 2. The second issue is that the AEC has not implemented bulk exclusion (Section 13A). Contrary to common belief (and probably contrary to the intentions of the authors of the Electoral Act), bulk exclusion can alter who wins a seat. The Act is hard to interpret on whether bulk elimination is compulsory. This is detailed in Section 3, together with examples to show why bulk exclusion can alter the election outcome. We recommend that bulk exclusion be deleted from the Electoral Act, matching the AEC’s current practice. ∗Andrew Conway is a member of the Secular Party. †Vanessa Teague is an advisory board member of Verified Voting, a non-governmental US organization working toward accuracy, integrity and verifiability of elections. 1 --- Page 2 --- As a third issue, the Electoral Act does not clearly define when the count ends, which is relevant because two termination conditions may apply simultaneously. This can leave ambiguity over who wins the last seat (under subsection (17)) and can alter the order in which candidates are considered to have been elected (under subsection 18). Suggested amendments are described in Section 4. 1.1 Prior work These results are similar to our prior discoveries of errors in the NSW [CBNT17] and ACT [CT] vote counts. The tiebreaking problems are similar to those identified by Wen [Wen10]. Most of our suggestions for simplifications are aligned with Gor´e and Lebedeva [GL16], whose suggestions for further improvements we also support. Other open-source STV counting programs include Grahame Bowland’s Dividebatur1 and its successor Dividebatur22, Lee Yingtong Li’s OpenTally3, and Milad Ghale’s formally verified STV4. Dividebatur2 is specifically implemented for the exact Senate rules, and performs tiebreaking correctly, but does not implement bulk exclusion. We have written extensively in the past about the importance of auditing the digitization process [BCT+19]. Although it is not directly relevant to this report, the importance of double- checking electronic processes is emphasised, so we reiterate that recommendation here: Recommendation 1. When the preference data files for Senate votes are published, there should be a rigorous statistical audit to check that they accurately reflect the paper ballots. This should be conducted in a way that allows Scrutineers to check both the algorithms and the data. The rest of this report details specific issues in the count. 1.2 Summary and purpose of this report We have identified one case in which the AEC’s software is different from the legislated rules, one in which it could be debated, and one in which the Electoral Act is ambiguous. These all need to be clarified before they make a difference to an election outcome. The Senate count, like other Single Transferable Vote legislation, is highly complex and de- pendent on a number of rather arbitrary choices. The ambiguity is the problem—almost any clarification defined in advance would suffice, as long as it was unambiguous and exactly followed by the AEC. For example, if the AEC agreed to publish its source code, the legislation could be amended to reflect it. (Although we would not generally recommend making legislation in this way, it would be harmless for the examples in this report.) Each section of this report includes specific recommendations for resolving the differences or clarifying the rules. Our aim in this report is to contribute to improving electoral legislation and process to achieve election results that are unambiguous and publicly verifiable. Everyone makes mistakes, and there is no reason that occasional errors should undermine public trust in the election, if there is a good process for checking and correcting the count. Improving processes to produce clear evidence of an accurate election outcome is of great benefit for public trust in Australian election outcomes. We encourage everyone to examine and double-check our results. Our code and examples are available at https://github.com/AndrewConway/ConcreteSTV 2 Three way ties: error in the WA 2016 Senate count When the lowest standing candidate is to be excluded, and there is a tie, there is an explicit rule for resolving it: Electoral Act, Section 273 31(b) ...if 2 or more continuing candidates have the same number of votes, those candidates shall stand in the poll in the order of the relative number of votes of each of those candidates at the last count at which each of them had a different number of votes, [our emphasis] with the continuing candidate with the greater or greatest number of votes at that count standing higher in the poll and the continuing candidate with the fewer or fewest number of votes at that count standing lower in the poll, but if there 1https://github.com/grahame/dividebatur 2https://github.com/grahame/dividebatur2 3https://yingtongli.me/git/OpenTally/ 4https://github.com/MiladKetabGhale/STV-Counting-ProtocolVerification 2 --- Page 3 --- Count HERCOCK, Marion FARGHER, Sara HENG, Henry 57 63 63 63 65 65 65 66 66 1 2 3 4 5..9 10,11 12..15 16..41 42..48 54 57 60 61 63 64 65 66 66 61 64 65 65 65 65 65 65 66 Table 1: Tiebreaking in the first 2016 WA Senate special election. Hercock was excluded, though Fargher had the lowest tally when each of the three candidates’ tallies were different (Count 4). has been no such count the Australian Electoral Officer for the State shall determine the order of standing of those candidates in the poll. There are very similar clauses in subsections (22) and (22b) for order of election and surplus distribution tie resolution. In the first WA 2016 Senate special count,5 (conducted on 7 March 2017, after Rod Cullerton was removed), this situation arose at Count 48, where there was a three-way tie for exclusion: M. Hercock, S. Fargher and H Heng all had 66 votes. Their tallies are shown in Table 1. The last count at which each of them had a different number of votes was Count 4, with 65, 61 and 63 respectively. Our reading of the legislation is that S. Fargher should therefore be excluded at Count 48. In the official count, the AEC excluded M. Hercock instead. There is a natural argument for this—at count 41 M. Hercock had a lower tally than either of the others, and there is no reason other than the legislation that this would not be a reasonable basis for tie resolution. Indeed, this approach has the advantage of resolving more situations than the “each of them had a different number” rule. It would be a reasonable thing to legislate, but it is not what the Electoral Act currently requires. Of course, different methods of resolving who is excluded can easily change the outcome of the election. As a very simple example of how this could affect who is elected, imagine that the number of vacancies was equal to the number of candidates remaining after the candidate in question was excluded. For this reason it is essential that the legislation and the official code be aligned. Recommendation 2. Either: • Correct the AEC’s official code to follow the legislation for tie resolution, or • Ask the AEC to publish the official code, including its tiebreaking rules, and amend the legislation to match it. It is also worth pointing out that sometimes the prior counts do not resolve the tie, and the Australian Electoral Officer is called upon to choose an order. In 2016 and 2019, it appears that all such decisions favoured candidates according to their position on the ballot, with lower candidates better favoured. This seems fair as the donkey vote otherwise favours higher candidates. This seems like an excellent policy which we fully endorse, and would suggest making part of the act. Recommendation 3. Formalize as legislation the apparent policy to, in the case of otherwise unresolved ties, favour candidates lower on the ballot. 3 Bulk Exclusions: omission in the 2016 and 2019 counts Section 273 subsection (13A) describes a bulk exclusion where multiple candidates are excluded together. It is commonly believed (and probably intended) that this can never alter the election outcome. In this section we show that that assumption is mistaken. The AEC does not appear to have implemented bulk exclusions since 2013, presumably in the mistaken belief that it cannot not alter the count. It is unclear whether bulk exclusion is at the AEC’s discretion or not. The Electoral Act Section 273 (13) states: 5The official Distribution of Preferences is at https://www.aec.gov.au/Elections/federal_elections/2016/ files/wa-senate-dist-prefs-2016fe-special-count-07032017.pdf 3 --- Page 4 --- (13) Where, after the counting of first preference votes or the transfer of surplus votes (if any) of elected candidates, no candidate has, or fewer than the number of candidates required to be elected have, received a number of votes equal to the quota: (a) the candidate who stands lowest in the poll must be excluded; or (b) if a bulk exclusion of candidates may be effected under subsection (13A), those candidates must be excluded; The logic seems unclear here—we are not lawyers and cannot say with confidence whether “those candidates must be excluded” would apply whenever (13A) permits it, but that would be one plausible interpretation. In this section we provide two simple examples to show that, in two different ways, bulk exclusion can alter the outcome. The first concerns the order of eliminations, the second is due to rounding. In each case, different people are elected depending on whether the bulk exclusion rule is applied. 3.1 Bulk Exclusions and order of elimination Subsection 13A (d) (i) does not have a clause requiring candidate C to satisfy a similar condition to 13A (b) (ii). This means that bulk exclusions can change the order of elimination. This can readily change the outcome of an election. A simple example is shown here with 6 vacancies, and votes as described in the table below: Preference List Occurrences L1, L2, W1, W3 L2 L3, L2, W1, W3 L4, W1, W2 L5, W4 L5, W5 L5, W6 L5, W7 L5 W7 W6 W5 W4 W3 W2 W2, W4 W2, W5 W2, W6 W1 2 3 4 6 1 2 3 4 10 400 400 400 400 400 388 6 4 2 400 Using the rules as applied by the AEC in 2013, as best we can reverse engineer, would result in the distribution of preferences shown in table 2, with bulk exclusion occurring on count 1: However, using the rules that seem to have been applied by the AEC in 2016, ignoring bulk ex- clusions, the order of elimination changes, and candidate W3 is elected instead of W2. Furthermore the order of election of candidates W4, W5, W6 and W7 is reversed, as shown in table 3 Curiously enough, using the rules apparently applied by the AEC in 2019, the same candidates are elected, but in a different order; see section 4.2. 3.2 Bulk Exclusions and rounding It is more obvious that Bulk Exclusions can change rounding, and this can change the result of the election. Rounding may seem like a small effect, but in 2016 Tasmania, there were 285 votes lost due to rounding, and the margin between the last two candidates was 143 votes. Here is a very simple demonstration of how the presence or absence of bulk exclusion can change rounding and thus who is elected in a 2 vacancy election. 4 --- Page 5 --- Count W1 W2 W3 W4 W5 W6 W7 400 400 400 400 400 400 Exclude L1, L2 400 400 400 400 400 400 1 400 2 +2 402 3 +4 406 Exclude L3 4 Exclude L4 5 Exclude L5 6 Exclude W3 400 +6 406 400 406 (cid:8)(cid:8)(cid:72)(cid:72)400 −400 0 406 406 406 406 400 400 400 400 400 400 400 400 400 +1 +2 +3 +4 404 401 403 402 401 402 403 404 L1 (cid:1)(cid:65)2 L2 (cid:1)(cid:65)3 −2 −3 0 0 L3 4 (cid:1)(cid:65)4 −4 0 L5 20 20 20 L4 6 6 (cid:1)(cid:65)6 −6 0 (cid:26)(cid:26)(cid:90)(cid:90)20 −20 0 Exh. TV +3 3 3 3 +10 13 +400 413 1 1 1 1 1 Table 2: Elected : W1, W2, W7, W6, W5, W4. Rules used : AEC2013. Elected candidates are colored purple once they have a quota. Count W1 W2 W3 400 400 400 W4 W5 W6 W7 400 400 400 400 400 400 400 400 400 400 400 400 400 400 400 400 400 L3 4 L1 (cid:1)(cid:65)2 L2 3 −2 +2 5 0 (cid:1)(cid:65)4 +4 −4 0 9 1 2 Exclude L1 3 Exclude L3 Exclude L4 5 Exclude L2 6 Exclude L5 7 Exclude W2 8 Surplus W4 400 4 +6 406 400 406 400 406 (cid:8)(cid:8)(cid:72)(cid:72)400 −400 0 406 406 (cid:1)(cid:65)9 −9 0 400 +6 406 406 406 406 400 400 400 400 400 400 400 400 +1 +2 +3 +4 401 404 403 402 +6 +4 +2 405 406 407 −1407 406 405 404 406 404 L4 6 6 (cid:1)(cid:65)6 −6 0 L5 20 20 20 20 (cid:26)(cid:26)(cid:90)(cid:90)20 −20 0 Exh. TV 1 1 1 1 1 1 1/407 +3 3 +10 13 +388 401 +1407 402 Table 3: Elected : W1, W3, W4, W5, W6, W7. Rules used : AEC2016. Elected candidates are colored purple once they have a quota. Superscripts are the number of papers corresponding to the given number of votes. 5 --- Page 6 --- Count 1 400 2 −100400 300 Q W1 W2 246 245 245 246 Surplus Q 3 Exclude L1, L2, L3 4 Exclude L1, L2, L3 300 300 245 +29 247 246 246 L3 3 L2 2 L1 1 Exh. Rounding TV +03 +03 +03 +97391 97 +6 103 (cid:1)(cid:65)1 −1 0 −03 −03 −03 (cid:1)(cid:65)3 −3 0 (cid:1)(cid:65)2 −2 0 103 +3 3 3 −2 1 1/4 1 1/4 Table 4: Elected : Q, W1. Rules used : AEC2013. Elected candidates are colored purple once they have a quota. Superscripts are the number of papers corresponding to the given number of votes. Count 1 400 2 −100400 300 Q W1 W2 246 245 245 246 L3 3 L2 2 L1 1 Exh. Rounding TV 3 +03 +03 +03 +97391 97 +1 98 2 3 2 (cid:1)(cid:65)1 −1 0 −03 3 3 (cid:1)(cid:65)2 −2 0 −03 (cid:1)(cid:65)3 −3 0 −03 98 +2 100 100 +3 103 103 +3 3 3 3 3 3 3 3 1/4 1 1/4 1 1/4 1 1/4 300 300 300 300 300 300 245 +03 245 245 +03 245 245 +03 245 246 246 246 246 246 246 Surplus Q 3 Exclude L1 4 Exclude L1 5 Exclude L2 6 Exclude L2 7 Exclude L3 8 Exclude L3 Table 5: Elected : Q, W2. Rules used : AEC2016. Elected candidates are colored purple once they have a quota. Superscripts are the number of papers corresponding to the given number of votes. Preference List Occurrences L3 L2 L1 W1 W2 Q, L3, W1 Q, L2, W1 Q, L1, W1 Q 3 2 1 245 246 3 3 3 391 Using bulk exclusion as in 2013 produces winners Q and W1 as shown in table 4. Not using bulk exclusion as in 2016 produces winners Q and W2 as shown in table 5. 3.3 Bulk Exclusion summary and conclusion These examples show that, as currently legislated, bulk exclusion can alter election outcomes, though it has not done so in practice so far. Bulk exclusion is an unnecessary complication for an electronic count. The ambiguity over whether or not to use it could result in a contested election outcome that is difficult or impossible to resolve. The best way to avoid this is to remove bulk exclusion from the Act. Recommendation 4. Remove Bulk Exclusion from the Act. This includes subsections (13A), (13B), (13C), portions of (13AA), and several definitions in (29). 6 --- Page 7 --- 4 Ambiguity in the Act: termination of the count There are three different ways a candidate can be elected: • by attaining a quota, • by being the higher of the two continuing candidates when there is exactly one unfilled seat (Subsection (17)). • by being a continuing candidate when the number of unfilled seats equals the number of continuing candidates, (subsection (18)) The Act does not explicitly define when the distribution of preferences ends—in particular, it is unclear when subsections (17) or (18) apply. Subsection (18) affects only order, not who gets elected. However, subsection (17) can affect who wins. Commonwealth Electoral Act 1918, Section 273, (17) In respect of the last vacancy for which two continuing candidates remain, the con- tinuing candidate who has the larger number of votes shall be elected notwithstanding that that number is below the quota, and if those candidates have an equal number of votes the Australian Electoral Officer for the State shall have a casting vote but shall not otherwise vote at the election. Unlike a computer program, which (usually) has an implicit but well-defined sequence of exe- cution steps, the Electoral Act simply lists a number of things that could happen (in subsections (9) through (18)) but does not specify which should be chosen when the conditions for two or more have been met. As a specific example, suppose one candidate (A) attains a quota (and hence a seat), leaving two other candidates, B and C, competing for a single last seat. Should subsection (17) apply immediately, electing whichever of B and C has a larger total, or should A’s surplus be distributed first? The legislation is not clear. Subsection (14) states (14) Any continuing candidate who has received a number of votes equal to or greater than the quota on the completion of a transfer under subsection (13) or (15) of ballot papers of an excluded candidate or candidates, as the case may be, shall be elected, and, unless all the vacancies have been filled, the surplus votes (if any) of the candidate so elected shall be transferred... The question is whether, “all the vacancies have been filled,” because (17) applies immediately, or instead (17) should not be applied until after this transfer has been completed. A similar ambiguity arises over exclusions (subsection (15)): when a candidate is excluded, should subsection (17) apply immediately, or should the excluded candidate’s preferences be distributed first? These decisions could affect who wins, because preference distribution could alter the relative rankings of the two continuing candidates. 4.1 Practical examples of when subsection (17) can alter the election outcome In practice, for all the examples we could find, the AEC seems to apply (17) only after all transfers have been completed. This seems like the right thing to do, and we believe the legislation should be amended to reflect (what seems to be) the AEC’s practice. Indeed, subsection (17) could simply be removed—it is unnecessary because the exclusion of the lower candidate immediately causes subsection (18) to apply (one candidate and one seat remaining). Resolving the ambiguity is important because this case arises frequently in practice. For exam- ple, consider the special count of the 2016 South Australian Senate votes, conducted in 2017 after the removal of Bob Day.6 The last steps are summarised in Table 6. At the end of Count 453, there are three remaining candidates: McEwen (ALP), Gichuhi (FFP), and Burgess (ON). Burgess has the least votes and is excluded, leaving only two candidates. If subsection (17) were applied imme- diately, McEwen would win (with the higher tally). But after all the preferences are distributed, Gichuhi wins. Similar examples arose in 2013 in NSW, Vic, WA and Tasmania. 6https://www.aec.gov.au/Elections/Federal_Elections/2016/files/sa-senate-dist-prefs-2016fe-special-count-13042017. pdf 7 --- Page 8 --- Count McEWEN GICHUHI BURGESS 453 Surplus KAKOSCHKE-MOORE 454–465 Exclude BURGESS 54099 52482 (cid:24)(cid:24)(cid:24)(cid:88)(cid:88)(cid:88)46684 65841 69442 0 Table 6: The last steps of the 2016 South Australian Senate special election count, conducted in 2017. If paragraph (17) was applied at count 453, as soon as Burgess was excluded, McEwen would win instead of Gichuhi. Count W1 W2 W3 W4 W5 W6 W7 400 400 400 400 400 400 400 400 400 400 400 400 400 400 400 400 400 400 400 400 L3 4 L1 (cid:1)(cid:65)2 L2 3 −2 +2 5 0 (cid:1)(cid:65)4 +4 −4 0 9 400 +6 406 406 400 400 400 400 400 400 400 400 +1 +2 +3 +4 404 401 403 402 (cid:1)(cid:65)9 −9 0 406 400 406 401 402 403 404 1 2 Exclude L1 3 Exclude L3 Exclude L4 5 Exclude L2 6 Exclude L5 7 Exclude W2 400 4 +6 406 400 406 400 406 (cid:8)(cid:8)(cid:72)(cid:72)400 L4 6 6 (cid:1)(cid:65)6 −6 0 L5 Exh. TV 20 1 1 1 1 1 20 20 20 +3 (cid:26)(cid:26)(cid:90)(cid:90)20 3 −20 +10 13 0 13 Table 7: Elected : W1, W3, W7, W6, W5, W4. Rules used : AEC2019. Elected candidates are colored purple once they have a quota. Recommendation 5. Remove subsection (17), or clarify that it applies only after all papers from surplus distributions and exclusions are transferred (which seems to be the AEC’s current practice). 4.2 Subsection 18 Subsection (18) does not affect who wins, but may affect their order—we do not know whether this matters in any politically-relevant way. In 2016, the AEC seemed to apply section 18 after finishing started exclusions and any surplus distributions that may thereby arise. For example, in the Queensland 2016 election, at count 830, candidate R. McGarvie was excluded, leaving 2 candidates and 2 seats. The exclusion was carried out in full (11 counts), and C Ketter was discovered to have a quota, leaving 1 candidate (M Roberts) and 1 vacancy. This candidate was not elected until count 841, when C Ketter’s surplus was distributed. Similar events happened in Victoria (count 814), and NSW (count 1054). A different thing occurred in WA 2016 (with R. Cullerton excluded) where on count 535 K. Muir was excluded, leaving 2 candidates and 2 seats. The first step of the exclusion was performed, at the end of which the remaining 2 candidates were both declared elected. Regardless, the transfer of votes for the first step of exclusion was always performed. In 2019 the rules seemed to change and the AEC applied subsection (18) immediately after identification of a candidate to be excluded, before transferring any votes. In 2019 NSW, count 429, K. McCulloch is excluded. This leaves 2 candidates, 2 vacancies. The preference distribution was aborted and no ballots were transferred in this count. Recommendation 6. Clarify when subsection (18) applies. The example used in Section 3.1 can be applied using the rules the AEC appeared to have used in 2019, aborting an exclusion after identification of the candidate to be excluded. This produces yet another different outcome from the same votes shown in table 7. This time only the order of election changes, with the order of election of candidates W4, W5, W6 and W7 being reversed. 8 --- Page 9 --- 5 Conclusion Everyone makes mistakes, and all software has bugs and security vulnerabilities.7 For election software, the crucial thing is not to make the software perfect (which is impossible) but to ensure an open and transparent process that allows the software, and its results, to be scrutinised and double-checked. It is a good thing when errors and vulnerabilities are found and fixed. We have found one error (in the implementation of tiebreaking) in the AEC’s official Senate counting software. This needs to be corrected before the next election, and we would be happy to help patch it if the source code is made openly available. Alternatively, the Electoral Act could be amended to match the AEC’s current algorithm—it does not really matter how ties are broken, as long as the algorithm is specified in advance. Modifying the Electoral Act to match the current algorithm would also be aided by opening the source code. The same applies to the other two ambiguities we identified in the Electoral Act: bulk exclusion and the order of application of subsection (17). The legislation is unclear, and needs to be clarified, either to match the AEC’s current code (if it is made available) or independently. A system that allows discretion over ambiguous rules, or runs a count different from the specified rules, may result in disputes at election time that are difficult or impossible to resolve. The purpose of this report is to avoid this outcome, by providing specific directions on correcting the AEC’s code (Section 2), realigning the AEC’s code with the unclear legislation (Section 3), and resolving ambiguitites in the legislation (Section 4). This should give very high confidence that the counting software is accurate and produces the unique outcome defined by legislation. It is easy to double-check the Senate counting software, because the software’s inputs (votes) and outputs (distributions of preferences) are both publicly available. This allows for the open public double-checking that is critical to democracy. (Of course it does not guarantee that all such problems will be found before they affect a count.) It is very helpful that the AEC makes this information publicly available. By contrast, it is not possible to double-check—so as to find and fix errors in—the software that converts paper ballots to digitized preferences. This leaves the AEC without a way of providing evidence of an accurate election outcome. A statistical audit of randomly-chosen paper ballots, in the presence of scrutineers, would allow the digitized preferences to be double-checked against their paper originals. This would give onlookers a chance to find and fix problems, or to see that there was good evidence that the number of discrepancies was small. 6 Summary of Recommendations Recommendation 1. When the preference data files for Senate votes are published, there should be a rigorous statistical audit to check that they accurately reflect the paper ballots. This should be conducted in a way that allows Scrutineers to check both the algorithms and the data. Recommendation 2. Either: • Correct the AEC’s official code to follow the legislation for tie resolution, or • Ask the AEC to publish the official code, including its tiebreaking rules, and amend the legislation to match it. Recommendation 3. Formalize as legislation the apparent policy to, in the case of otherwise unresolved ties, favour candidates lower on the ballot. Recommendation 4. Remove Bulk Exclusion from the Act. This includes subsections (13A), (13B), (13C), portions of (13AA), and several definitions in (29). Recommendation 5. Remove subsection (17), or clarify that it applies only after all papers from surplus distributions and exclusions are transferred (which seems to be the AEC’s current practice). Recommendation 6. Clarify when subsection (18) applies. 7Even our software. We found the AEC tiebreaking error when we were re-implementing our version of the Senate count and noticed we had made the same mistake. 9 --- Page 10 --- References [BCT+19] Michelle Blom, Chris Culnane, Vanessa Teague, Damjan Vukcevic, Andrew Conway, Rajeev Gor´e, and Peter Stuckey. Submission to the inquiry into the conduct of the 2019 federal election, 2019. https://www.aph.gov.au/DocumentStore.ashx?id= a47d4582-b11f-4731-b87a-852ddab7f1c3&subId=670312. [CBNT17] Andrew Conway, Michelle Blom, Lee Naish, and Vanessa Teague. An analysis of New South Wales electronic vote counting. In Proceedings of the Australasian Computer Sci- ence Week Multiconference, pages 1–5, 2017. https://arxiv.org/pdf/1611.02015. pdf. [CT] [GL16] Andrew Conway and Vanessa Teague. Errors in the act’s electronic count- ing code. https://github.com/SiliconEconometrics/PublicService/raw/master/ CountVotes/2020%20Errors%20In%20ACT%20Counting.pdf. Rajeev Gor´e and Ekaterina Lebedeva. Simulating stv hand-counting by computers In International Joint Conference on Electronic Voting, considered harmful: Act. pages 144–163. Springer, 2016. https://openresearch-repository.anu.edu.au/ bitstream/1885/113153/2/01%20Gore%20and%20Lebedeva%20Simulating%20STV% 20hand-counting%202017.pdf. [Wen10] Roland Wen. Online Elections in Terra Australis. PhD thesis, University of New South Wales, Sydney, Australia, 2010. http://unsworks.unsw.edu.au/fapi/datastream/ unsworks:10365/SOURCE02. 10
This document, titled "Corrections to the AEC’s counting code, and recommended amendments to the Electoral Act, for Senate Counting & Scrutiny" by Dr Andrew Conway and A/Prof Vanessa Teague (August 31, 2021), identifies discrepancies between the Australian Electoral Commission (AEC)'s Senate election counting software and the requirements of the Electoral Act, along with ambiguities within the Act itself.
Key Issues Identified:
* Tie-breaking Inconsistency: The AEC's software applied an incorrect tie-breaking rule for 3-way ties (e.g., in the 2016 WA Senate election), leading to candidates being excluded in the wrong order, contrary to Electoral Act Section 273 31(b). This deviation could potentially alter election outcomes.
* Bulk Exclusion Omission: The AEC has not implemented bulk exclusions (Electoral Act Section 273 (13A)) since 2013, under the presumed, but mistaken, belief that it does not affect the count. The report demonstrates that bulk exclusions can, in fact, change election outcomes due to altered elimination order and rounding effects. The Act's language on bulk exclusion is ambiguous regarding its mandatory application.
* Ambiguity in Count Termination: The Electoral Act lacks clarity on when the count terminates, particularly concerning the simultaneous application of subsections 273 (17) (electing the higher of two remaining candidates for the last seat) and (18) (electing remaining candidates when seats equal continuing candidates). This ambiguity can affect who wins the last seat or the order in which candidates are declared elected.
Recommendations:
The report proposes several legislative and procedural changes:
* Correct the AEC's official code to comply with legislated tie-breaking rules, or alternatively, publish the code and amend the legislation to match it.
* Formalize the policy of favoring candidates lower on the ballot for otherwise unresolved ties.
* Remove bulk exclusion provisions from the Electoral Act to eliminate complexity and ambiguity.
* Clarify or remove subsection 273 (17) to ensure it applies consistently (preferably after all transfers are completed, aligning with AEC's current practice).
* Clarify when subsection 273 (18) applies.
* Implement a rigorous statistical audit of digitized preference data files against paper ballots to ensure accuracy and transparency.
Relevance to FOI Request:
This document is the "subsequent 2021 report by Conway and Teague" directly referenced in the FOI request overview. It provides the detailed findings concerning "accuracy, methodology, and identified issues with the Australian Electoral Commission's (AEC) Senate election counting algorithm and associated scrutiny processes." Specifically, it substantiates the FOI request's points regarding:
* The AEC's software using tie-breaking rules inconsistent with the Electoral Act.
* The non-implementation of bulk exclusions and their potential to alter outcomes.
* Ambiguities in the Electoral Act concerning the count's termination.
* It details the proposed "legislative and procedural recommendations," including publishing the AEC's code, correcting tie-breaking, and clarifying statutory definitions, which the AEC subsequently responded to. As such, it is central to understanding the discrepancies investigated by the FOI request.
Document 6 - Corrections to the Senate Counting algorithm [SEC=OFFICIAL] (A1861209).pdf (pdf)
Download file--- Page 1 --- From: To: Cc: Subject: Date: Attachments: Kristen Wild Vanessa Teague andrewstvcount@greatcactus.org; Tom Rogers; Thomas Ryan RE: Corrections to the Senate Counting algorithm [SEC=OFFICIAL] Tuesday, 19 October 2021 11:28:00 AM 18 10 2021 - Dr Andrew Conway and Prof. Vanessa teague - Senate Counting and Scrutiny.pdf image001.gif image002.gif image003.jpg Good morning Please find the attached response from Mr Rogers. Regards Kristen Kristen Wild | Personal Assistant to the Commissioner Executive Leadership Team Australian Electoral Commission From: Vanessa Teague <Vanessa.Teague@anu.edu.au> Sent: Tuesday, 31 August 2021 3:40 PM To: Tom Rogers Cc: andrewstvcount@greatcactus.org Subject: Corrections to the Senate Counting algorithm ; Thomas Ryan CAUTION: This email originated from outside of the Australian Federal Government. Do not click links or open attachments unless you recognise the sender and know the content is safe. Dear Mr Rogers and Mr Ryan, Please find attached our report on the AEC's Senate count. We have found one instance in which the software breaks ties in a way that is not what is specified in the Electoral Act. We have also noted some other ambiguities that need to be resolved either by amending the Act or by otherwise committing to the exact counting algorithm. We would be very happy to help you correct the code - you can see our implementation here https://github.com/AndrewConway/ConcreteSTV. We could easily make the necessary adjustments to the official code if it was similarly published online. Also, we support recent proposals to audit randomly selected paper ballots at the end of the digitizing process to give everyone evidence of the accuracy of the process. We would be happy to help the AEC implement these measures securely and transparently. Yours Sincerely, Vanessa Teague and Andrew Conway. Vanessa Teague --- Page 2 --- CEO, Thinking Cybersecurity Pty. Ltd. A/Prof (Adj.), Research School of Computer Science The Australian National University | Canberra ACT 2601 E: vanessa.teague@anu.edu.au W: https://researchers.anu.edu.au/researchers/teague-v CRICOS Provider #00120C
This document is an email chain initiated by Vanessa Teague and Andrew Conway (August 31, 2021) to Australian Electoral Commission (AEC) Commissioner Tom Rogers, formally submitting their report on the AEC's Senate count. In this submission, they highlight a finding that the AEC's software "breaks ties in a way that is not what is specified in the Electoral Act" and notes other ambiguities requiring resolution through legislative amendment or a clarified algorithm. They offer assistance with code correction and support auditing paper ballots. The document's later email (October 19, 2021) from Kristen Wild (AEC) to Vanessa Teague then forwards Mr Rogers' (AEC) official response to their report.
Relevance to FOI Request:
This document is directly relevant as it provides primary evidence of the 2021 Conway and Teague report's official submission to the AEC. It explicitly details one of the "significant discrepancies" identified by the report, namely the inconsistent tie-breaking rules, a core focus of the FOI request. Furthermore, it confirms the AEC's subsequent engagement and response to the report's findings, directly addressing the FOI's scope regarding identified issues with the counting algorithm and the AEC's actions in response.
Document 7 - 18 10 2021 - Dr Andrew Conway and Prof. Vanessa teague - Senate Counting and Scrutiny (A1860875).pdf (pdf)
Download file--- Page 1 --- --- Page 2 ---
The provided document is empty. Therefore, no summary of its content can be made, and its relevance to the FOI request regarding the AEC's Senate election counting algorithm and scrutiny processes cannot be determined from its contents.