Ahead of his Conservative Party conference speech outlining how the government intends to axe £10bn from the welfare budget, the chancellor says the wealthy will bear the "broadest burden".
Mr Osborne is expected to use his speech this morning to describe which benefits payouts are to be cut or curtailed by 2016-2017.
Among those to be cut will be housing benefit for the under-25s, as previously suggested by David Cameron. Under the plans, the majority of the £1.8bn paid 380,000 under-25s will be cancelled.
But the chancellor said he is not targeting the poor, and that the wealthy had paid more under every one of his budgets.
"Let me be clear, those with the broadest shoulders must bear the broadest burden, but our perception of fairness - and this is perhaps where we differ from the Labour Party - also extends to the welfare system," he told the BBC.
"We also think it is unfair that person leaves their home early in the morning and they pull the door behind them and they are going to do their job and they look at their next-door neighbour, the blinds are down and that family is living a life on benefits.
"That is unfair as well and we are going to tackle that as part of tackling this country's economic problems."
Mr Osborne has secured an agreement from Iain Duncan Smith, the work and pensions secretary, to implement the cuts.
However, the announcement could set the Conservative Party on a collision course with their coalition partners after Nick Clegg told the Liberal Democrat conference that he would not allow such extensive cuts to the welfare system.
The deputy prime minister said last month that he would not allow "wild suggestions" of a £10bn cut. Chief secretary to the Treasury Danny Alexander told delegates: "We simply will not allow the books to be balanced in a way that hits the poorest hardest."
However, Mr Osborne will say that each of his budgets has increased overall taxes on the very wealthiest and that, despite cutting the top rate of tax from 50p to 45p, the rich will pay a greater share of the UK's tax revenue in each year of the current government compared to the Labour Party's 13 years in power.