Fees are a payment for the transfer of funds over the bitcoin network and compensate miners for their contribution to the system and protection from double spendings.
The number of transactions that can be included within a block is limited, since it can only weight a maximum of about 2MB. The fee that has to be paid is proportionate to the available space within a block and the size of the transaction, expressed in bytes and is not affected by the amount of funds to be transferred. Depending on the information included, the different operations of a transaction will have a different weight in bytes. For instance, transactions sending more information, such as sending bitcoin to more people at the same time will have a higher weight and will therefore have to pay a higher fee.
The fee market can be very competitive; since fees are paid to miners, they will prioritize the transactions that pay a higher fee per byte. The higher the fee set and the more likely a miner will include the transaction in a block, confirming it.
The minimum fee necessary to confirm a transaction can vary, depending on the number of people transacting on the Bitcoin network at a specific time. The higher the number of users, the more competition to be included in a block, resulting in higher fees .The optimal fee is usually calculate by your wallet, depending on the level of priority desired. In case of transactions having a higher priority, a higher fee will be set, guaranteeing a faster confirmation.
As an example, we hypothesize that a miner can only include up to 2000 transactions in a block: if there are only 1000 users that wish to transact on the bitcoin network there is minimal competition and all transactions, even those with minimal fees, will be confirmed.
On the other hand, if there are 4000 people that wish to transfer bitcoin, there will be high competition to figure among the 2000 that will get their transactions included within a block. In this scenario, the miner will simply choose the 2000 transactions paying higher fees, creating a sort of auction where all users try to set a high enough commission to be included within a block. This process happen in the background and is not carried out manually by the users. Within each Bitcoin wallet there are specific components, tasked with providing the ideal commission depending on the state of the network at a given time.