Hello,
Originally Posted by werehk Why the answer is not zero but 24N? 
There are four forces acting on the wedge : the force exerted by the ground $\displaystyle \vec{N}_{\text{ground}}$, its weight $\displaystyle \vec{W}_{\text{wedge}}$, the force $\displaystyle \vec{F}$ and the force $\displaystyle \vec{N}$ exerted by the block. We are told that the wedge is at rest so, using Newton's second law :
$\displaystyle \vec{N} + \vec{W}_{wedge} + \vec{N}_{\text{ground}} +\vec{F}=\vec{0}$
As "all contact surfaces are assumed to be smooth", the force exerted on the wedge by the ground is vertical so the projection of the previous relation on the xaxis (horizontal) gives us
$\displaystyle \vec{N}\cdot \vec{x} + \underbrace{\vec{W}_{wedge}\cdot \vec{x}}_0 + \underbrace{\vec{N}_{\text{ground}}\cdot \vec{x}}_0 +\underbrace{\vec{F}\cdot \vec{x}}_{\pm\\vec{F}\}=\vec{0}$
hence
$\displaystyle \\vec{F}\=\vec{N}\cdot \vec{x}=\ \vec{N}\\times \underbrace{\\vec{x}\}_1 \times \cos \theta_1 \implies \boxed{\\vec{F}\=\ \vec{N}\ \times \cos \theta_1}\,\,\,\,\,\,\,(1)$
Now, what is $\displaystyle \vec{N}$ ? Let's take a look at the attached picture. The two forces which are acting on the block are its weight $\displaystyle \vec{W}=m\vec{g}$ and the force exerted by the wedge which is $\displaystyle \vec{N}$. Again, as "all the surfaces are assumed to be smooth", we get $\displaystyle \vec{N}\perp (AC)$. When the block slides, we know it moves along a straight line parallel to $\displaystyle (AC)$ so the direction of its acceleration is parallel to $\displaystyle (AC)$ too. Using Newton's second law we get
$\displaystyle m\vec{a} =\vec{W} \vec{N}=m \vec{g}\vec{N}$
The projection of this relation on an axis $\displaystyle z$ perpendicular to $\displaystyle (AC)$ gives us
$\displaystyle \underbrace{m\vec{a}\cdot \vec{z}}_0=\vec{W}\cdot \vec{z}\underbrace{\vec{N}\cdot \vec{z}}_{\pm\\vec{N}\} \implies \boxed{\\vec{N}\=mg\cos \theta_0}\,\,\,\,\,\,\,(2)$
Putting this together with (1) :
$\displaystyle \\vec{F}\ = mg\cos \theta_0\cos \theta_1=5 \times 9.81\times \frac{40}{\sqrt{40^2+30^2}}\times \frac{30}{\sqrt{40^2+30^2}}=23.5\,\text{N}$
For force applied F, where comes a force making the wedge stationary?

I'm not sure to understand what you're asking. If you want to know how can one exert the force $\displaystyle \vec{F}$ on the wedge then one solution is to put the wedge against a wall, the wall being on the right side of the wedge (any other object fixed to the ground will do).