If I understand your problem correctly, and it is a little bit vague, is that you have a number of DjVu files that you would like to convert to PDF. The DjVu's that you want to convert have been generated from a scanner and thus the underlying data format is a bitmap/raster/picture graphics type format.
You are incorrectly assuming the capabilities of the PDF format (Adobe Spec) (Wikipedia Article). The PDF format is basically a simple mark-up language that describes how to format and place text and images on a page, do basic vector drawing, and also has the ability to have picture data encoded with in it. The text and page description mark-up language is usually then encoded with either the LZW or Flate compression algorithms. The picture data is stored as either JPEG (Page 39), TIFF (Page 71) or GIF (Page 842).
The PDF format does not use any novel technology like the DjVu format (DjVu Spec) (Wikipedia Article) does. Basically DjVu scans an entire document and looks for things that it can combine together, and then leave a reference to where each object appears. The DjVu format is much more clever than PDF in this regard. In a sense the DjVu format is more similar to a JPEG than it is to PDF when you look at the how the algorithm operates across an entire multi-page document.
A PDF document created by a scanner is just a series of pages defined to be a certain size, with a JPEG pasted into each page. You could run the PDF Data through an OCR (Optical Character Recognition) process, and recreate the document in a text format, and this would hugely reduce the size of the PDF document. The PDF document does not have OCR built into the format, but some readers, such as the Adobe Reader, has built-in OCR that lets you search within a JPEG-type PDF document.
I also understand that the Adobe Acrobat does have OCR processing capability. There are other programs, OmniPage comes to mind, as well as there are a number of open source OCR systems out there.