The approach to handling a congregation's expenses can vary greatly based upon the size of the group. With a small church, and groups that are just starting out, a very informal approach is possible. Large groups, however, definitely need a formal plan.
Do not pass a collection vessel (e.g. plate or box) at church services. This may sound odd to some, but there is no Biblical example of this practice (1Corinthians 16:2 talks about "storing up" things, not putting them in a collection on that day).
Concerning finances in the Old Testament, when people brought money to God's temple, Jehoiada instructed the priests to put it in a box with a hole in the top (2Kings 12:9). A box or some other container with a hole on top is a good way to collect offerings.
Church people have the right to give without others knowing when or how much they give (Matthew 6:3). Personal finances should be kept in the strictest confidence, known only to the people who maintain the records, and shared only with those who gave, in case there is a question.
Records should not be used by religious leaders to check up on people or to determine whether someone is 'worthy' to perform certain church responsibilities (e.g. serving like a deacon, etc.).
The congregation has a right to know not only how much money is received but also how it is spent. Members whose spiritual gift is "giving" need to know if their giving is bearing fruit. They cannot know if they do not have any way to see how the money is being spent.
Very small congregations can sometimes be run without the collection money. For example, one member will pay for the music equipment, one for the hymnals, one for the refreshments, and so on. Others might serve by letting the congregation use their home, vehicles or other resources. If a rental hall is used, two or three families might agree to alternate months paying the rent on the hall.
Still other members might agree to pay for evangelism and other efforts. Each member is free to handle the tax aspects of their work in whatever manner they choose. The congregation can stay out of it completely, or if necessary, write letters to members at the end of the year acknowledging the money spent on behalf of the church.
Sometimes there is a need to pay cash for books, youth class supplies, evangelism needs, and other expenses. The people who have the time to do the work may simply not have the finances to do it. If the brethren are willing to give offerings in cash, then the church money can simply be kept in a box in some safe place and expended by people who are authorized to do so. While this method may sound primitive, it has worked for thousands of years, even by our Messiah and His Apostles (John 13:29).
Bills that must be paid by check or credit card can be paid by a member and then reimbursed with cash. While paper records of all transactions should be kept so that members can know what their church is doing, the current assets of the congregation can be known by simply counting the money in the box. This method avoids bank service charges, check purchases, accidental overdrafts and associated fees, etc.
Please note that the above suggestions will likely not work for cases where the fellowship needs to own property, pay a large variety of bills or pay church workers for their time. If a congregation breaks up and the congregation owns the assets, it may take some time to decide how to divide the assets. Whereas, when individuals own the assets, they will usually take them to whatever new group they join and use them.
The finances of a local congregation can be kept manually or by computer. Record keeping will be easier for someone with accounting experience, but anyone with a good sense for math can do it. A simple journal listing the date, purchaser, what was purchased and so on will work.
Church finance systems that are more complex will classify each expense into categories or accounts: hall rental, sound system, music, utilities, office supplies, food, janitorial supplies, evangelism, etc. These more complex systems are much easier to do by computer. Numerous church accounting programs are available for a reasonable cost. Whatever kind of system you have should be balanced every month.
If no record is kept of the total amount of offerings received, it will not be possible to tell if any of it is lost or stolen. Fellowships should be good stewards of what they are given. While money is safer with an honest man who keeps no records than it is with a room full of dishonest accountants with piles of records, most people today expect religious organizations to keep some kind of accurate records (2Corinthians 8:21).
A church that emphasizes anonymous offerings may choose not to keep records of each individual's giving. This increases privacy, but it can also produce some difficulties with finances. For example, it will not be possible to give people a year-end total of their offerings. People should be able to keep their own records of their offerings, but many are simply used to congregations doing it. Pray for wisdom to know what is best for you.