Dfyjkt

The rule for grappling allows me to replace an attack with an attempt to grapple.

The Two-weapon Fighting rule allows me to use a bonus action to attack using my off-hand.

Can I replace the off-hand attack with a grapple/shove attempt?

Tl;dr: Nope. Not as your primary action, and not as your Two Weapon Fighting attack. Try using haste instead!

There are two places in the rules for you to consult - the rules for Two-Weapon Fighting in the Combat section of the Player's Handbook:

When you take the Attack action and attack with a light melee weapon that you’re holding in one hand, you can use a bonus action to attack with a different light melee weapon that you’re holding in the other hand. You don’t add your ability modifier to the damage of the bonus attack, unless that modifier is negative.

If either weapon has the thrown property, you can throw the weapon, instead of making a melee attack with it.

...and the rules for Grappling directly adjacent to them:

When you want to grab a creature or wrestle with it, you can use the Attack action to make a special melee attack, a grapple. If you’re able to make multiple attacks with the Attack action, this attack replaces one of them. The target of your grapple must be no more than one size larger than you and must be within your reach. Using at least one free hand...

There are two reasons that you can't make an "off hand attack" that initiates a grapple. Firstly, the rules for grapple specify that you can only use the Attack action (note the capital) to initiate a grapple, while Two Weapon Fighting allows you to use a bonus action to make an additional attack with a light weapon. The second reason is that Two Weapon Fighting requires you to be holding a light weapon in each hand, while grappling requires a free hand.

However, you may not have to abandon hope of Barbarian attacking+grappling shenanigans with your high Athletics score - provided you can convince a friendly caster near you to provide you with a cast of the haste spell.

Choose a willing creature that you can see within range. Until the spell ends, the target's speed is doubled, it gains a +2 bonus to AC, it has advantage on Dexterity saving throws, and it gains an additional action on each of its turns. That action can be used only to take the Attack (one weapon attack only), Dash, Disengage, Hide, or Use an Object action.

When the spell ends, the target can't move or take actions until after its next turn, as a wave of lethargy sweeps over it.

Unfortunately, an attempt to grapple is not a "weapon attack", and so you can't use your Haste action to grapple them. However, there's nothing to say you can't use your regular action to grapple them, and them hit them upside the head with your Haste action.

No, you cannot

Let's see the relevant rules:

Two-Weapon Fighting [..] you can use a bonus action to attack with a different light melee
weapon that you're holding in the other hand.

Grappling [..] you can use the Attack action to make a special melee attack, a
grapple. [..] Using at least one free hand, you try to seize the
target by making a grapple check (PHB 195, emphasis mine)

Since an off-hand attack does not use the Attack action and also because you do not have a free hand when dual wielding, you cannot use two-weapon fighting to grapple as a bonus action.

No. You can grapple in lieu of making one attack during an Attack action.

When you want to grab a creature or wrestle with it, you can use the Attack action to make a special melee attack, a grapple. If you’re able to make multiple attacks with the Attack action, this attack replaces one of them.

In other words: if a rule says you can make an Attack action, you can replace one of those attacks with a Grapple. If a rule just says you can make an attack, you can only make an attack.

Since Two Weapon Fighting only allows you to "use a bonus action to attack with a different light melee weapon", it isn't an Attack action or any part thereof and isn't valid for grappling with. The same goes for the Barbarian Berserker's Frenzy ability, the War Cleric's War Priest ability, the Monk's Martial Arts and Flurry of Blows, and so on.

By contrast, the Fighter's Action Surge lets you actually take an action, so you could use that to make a second Attack Action and perform a grapple during it.

There's also the difficulty of having two weapons in hand and still having a free hand, but there are ways around that, such as being polymorphed into a multi-armed creature or using a class feature that allows a bonus-action attack rather than Two Weapon Fighting per se.