Finetuning Gemma-2 to Preserve the Words and Thought of the Italian Enlightenment philosophy

by Stefano Trinchero
Ideas
AI NLP History Lang [eng]

Can a language model become an innovative tool for preserving a specific cultural context? This project is an attempt to fine-tune Gemma 2B to preserve a system of thought and the specific language used to articulate it.

This text was originally published on Kaggle as a submission for the contest Google - Unlock Global Communication with Gemma

0. Overview {#0-overview}

The primary motivation for this project stems from a fundamental question: can a language model become an innovative tool for preserving a specific cultural context? Beyond traditional methods such as archiving texts, critical analysis, and academic studies, can we create a new form of preservation that safeguards the words and ideas of a philosophical movement? Could this tool, in effect, transport these concepts to the present day, enabling them to be used and compared with contemporary matters?

In short, the goal of this project is to fine-tune Gemma 2B in an attempt to preserve a system of thought and the specific language used to articulate it.

Specifically, I chose to focus on the Italian Enlightenment, a philosophical and political movement that fascinates me due to its contribution to Italian social and civic progress. This movement emphasized the value of education, scientific inquiry, and addressing the needs of the people. It was deeply rooted in reality, politics, reform, and the social context of its time.

From Wikipedia: “The Enlightenment in Italy (Italian: Illuminismo italiano) was a cultural and philosophical movement that began in the second half of the eighteenth century, characterized by the discussion of the epistemological, ethical, and political issues of the Enlightenment thought of the eighteenth century.”

I thought that if I could fine-tune a language model to build an AI capable of replicating the syntax and vocabulary of 18th-century Italian and the structure of the Enlightenment thought, maybe I could then use it to analyze the current reality through an Enlightenment framework.

1. The collection of sources {#1-the-collection-of-sources}

The Italian Enlightenment primarily developed in two cities: Naples and Milan. When creating the dataset, I made an effort to balance texts from Neapolitan authors (such as Gaetano Filangeri and Mario Pagano) with those from Lombard authors (such as Cesare Beccaria, Alessandro and Pietro Verri - for those who speak Italian, a great online resource about Lombard Enlightenment is available at https://illuminismolombardo.it/).

In selecting the text fragments used for training, I aimed to include the key themes explored by Enlightenment philosophers. This was done to ensure the language model would be prepared to address a wide range of topics, including:

  • Justice
  • Environment
  • Economics
  • Customs and Manners
  • Work
  • Science
  • Health
  • Art and Culture
  • Education and Schooling
  • Architecture

The texts utilized for training were the following:

  • Francesco Algarotti, “Dialoghi sopra l’ottica neutoniana”, (1764): a popularized explanation of Newton’s theories of optics presented in a dialogue format.
  • Francesco Algarotti, “Dialogo sopra la lingua francese”, (1750). This is a short essay about the different historical development of Italian and French languages. It is a work of brilliant popularization (“divulgazione brillante”), a literary genre which aimed to simplify and make engaging the explanation of complex subjects.
  • Francesco Algarotti, “Saggio sopra la necessità di scrivere nella propria lingua”, (1750): an essay about the importance of authors writing in their native language for clarity and cultural expression
  • Cesare Beccaria, Dei delitti e delle pene, (1764). One of the most famous works of the whole Italian Enlightenment movement, this is a treatise on the state of the judiciary system that advocates for a more humane and rational criminal justice system, emphasizing principles of fairness, proportionality, and the prevention of crime through education rather than brutal punishment. This work became a foundational document for legal reform across Europe and Italy in particular.
  • Cesare Beccaria, “Della Commedia: presso i Greci, i Latini e gl’Italiani”. The book examines the history and evolution of comedy across three distinct cultures: the ancient Greeks, the Romans, and the Italians of his own time.
  • Francesco Algarotti, “Viaggi di Russia”, (1739): record of a 1739 journey by the author to Saint Petersburg and other Northern European cities.
  • Gaetano Filangeri, “La Scienza della Legislazione”, (1780-1788). The essay examines the systematic principles of a practical science designed to guide legislative reforms, with individual citizen happiness serving as the utilitarian foundation for a well-functioning state.
  • Ferdinando Galiani, ”* De’ doveri de’ principi neutrali verso i principi guerreggianti, e di questi verso i neutrali*”, (1782). The text is divided into two books: the first discusses the duties of neutral parties (particularly regarding commerce and contraband goods), deriving these duties from the principles of morality and law; the second addresses the related norms based on Reason of State.
  • Francesco Mario Pagano, Lettere: a collection of letters on the topic of justice and the organization of the state, written by one of the most important philosophers and jurists of the Neapolitan Enlightenment.
  • Pietro Verri, “Storia di Milano”, (1783): a detailed history of the city of Milan from its origins to the 18th century.
  • Pietro Verri, “Diario militare”, (1759): a personal diary documenting the author’s experiences and thoughts while serving in the military.
  • Pietro Verri, “Pensieri di un buon vecchio che non è letterato”, (1796): a late work by the philosopher, economist, historian, and writer known as the founder of the Milanese Enlightenment.
  • Pietro Verri, “Discorsi sull’indole del piacere e del dolore, sulla felicità e sulla economia politica”, (1781): a collection of philosophical, economic, and political discourses by one of the most prominent figures of this philosophical movement.
  • Various Authors, “Il Caffé”, (1764-1766). Il Caffé was an influential journal founded by Pietro and Alessandro Verri, and published in Milan from 1764 to 1766. The magazine was a vibrant platform for intellectual debate, social commentary, and the dissemination of Enlightenment ideals in Italy. This text represents the most important part of the dataset, both in terms of size and the variety of themes covered, as well as the way in which the ideas are presented.

2. Text processing {#2-text-processing}

Most of the texts were sourced from Liber Liber, a leading repository for Italian public domain books. Some texts were readily available in TXT format, while others were downloaded as PDFs and then converted using OCR (Optical Character Recognition).

After obtaining the plain text of a book, I processed it using the spaCy library with the Italian large model (it_core_news_lg). This involved tokenizing the text and splitting it into manageable sentences.

The following code demonstrates how a sample text file, enlightenment-text-example.txt, is processed. This file, an extract from Francesco Algarotti’s “Viaggi di Russia”, is included in the dataset attached to this notebook. The primary function of the script is to segment the text into chunks of sentences and prepare it for use in a dataset.

Here’s a breakdown of the processing steps:

  1. Text Loading and Cleaning:
    • The script loads the text content from the file specified by the file_path variable.
    • It then cleans the text by removing newline characters and extra spaces using the clean_text function.
  2. Text Segmentation:
    • The segment_text_into_chunks function uses the spaCy library to first tokenize the text into sentences. The desired length of each segment (in number of sentences) is set by the segment_length variable.
    • It then groups the sentences into segments. If any sentences remain at the end of the text that are fewer than the specified segment_length, these remaining sentences are grouped together into a final segment.
    • Each segment will be a string containing a variable number of sentences (up to segment_length sentences).
  3. Dataset Creation and CSV Output:
    • Each segment is added to the dataset list as a dictionary with two keys: "completion" (the segment itself) and "source", which, in this example, is hardcoded to “Francesco Algarotti, Viaggi di Russia”. I used the source column to track the provenance of the fragments, since I later assembled all the .csv files into a single file.
    • Finally, the script writes the dataset to a CSV file named example_segments.csv. This CSV file contains two columns: completion and source.

This script provides an example for creating a dataset from a text file, by creating segments of sentences using the segment_length variable.

In this example, the text is segmented into chunks of four sentences (segment_length = 4), but you can easily adjust the segment_length variable to create segments of different lengths.

For my experiments, I personally used two different segment_length values: segment_length = 2 to generate shorter, more aphoristic-sounding segments, and segment_length = 4 for broader, more structured segments.

This is the code I used to process the texts:

!pip install spacy -q
!python -m spacy download it_core_news_lg -q

import spacy
import os
import csv

# Define file paths
file_path = '/kaggle/input/italian-enlightenment-q-and-a/enlightenment-text-example.txt'
output_file = '/kaggle/working/example_segments.csv'

# Function to clean the text
def clean_text(text):
    text = text.replace("\n", "").strip()
    text = " ".join(text.split())
    return text

# Load Spacy model
nlp = spacy.load("it_core_news_lg")

def segment_text_into_chunks(text):
    segment_length = 4
    doc = nlp(text)  # Analyze text
    sentences = [sent.text.strip() for sent in doc.sents]  # Extract sentences

    segments = []
    current_segment = []

    for sentence in sentences:
        current_segment.append(sentence)
        if len(current_segment) == segment_length:  # Add segment after four sentences
            segments.append(" ".join(current_segment).strip())
            current_segment = []

    # Add left sentences
    if current_segment:
        segments.append(" ".join(current_segment).strip())

    return segments

# List to contain data
dataset = []

# Process single file
try:
    with open(file_path, 'r', encoding='utf-8') as f:
        text = f.read()

    # Clean text
    processed_text = clean_text(text)

    # Create segments made of four sentences
    segments = segment_text_into_chunks(processed_text)

    # Create examples for the dataset
    for segment in segments:
        example = {
            "completion": segment,
            "source": "Francesco Algarotti, Viaggi di Russia"
        }
        dataset.append(example)


    # Write the dataset to a CSV file
    with open(output_file, 'w', encoding='utf-8', newline='') as csvfile:
        fieldnames = ["completion", "source"] # Define the CSV header
        writer = csv.DictWriter(csvfile, fieldnames=fieldnames)

        writer.writeheader() # Write the header row
        writer.writerows(dataset) # Write the rows of data

    print(f"Dataset created and saved in {output_file}")


except FileNotFoundError:
    print(f"Error: File not found at {file_path}")
except Exception as e:
    print(f"An error occurred: {e}")
abs

3. Assembling the dataset {#3-assembling-the-dataset}

Following the automated processing of the text, I manually reviewed the dataset. This involved removing segments that were not semantically meaningful and merging sentences that, based on their content and completeness, naturally belonged within the same segment. I also eliminated all the notes and entire passages written in latin, french or other foreign languages.

My goal was to create a comprehensive list of text fragments. To each fragment, I would then add a potential prompt, designed to elicit a specific response. This process would ultimately create a dataset composed of two elements: prompts, formulated in contemporary Italian, and completions, consisting of fragments from Enlightenment texts written in 18th-century Italian.

As mentioned in the introduction, this dataset will be used to fine-tune Gemma 2, training it to respond to questions using the language and the thought patterns characteristic of the Italian Enlightenment.

The objective was not to cite passages from the original texts or to mention the sources of information. Instead, it was to recreate a language and a collection of concepts, and to use them to generate new texts.

Let me give you some examples from the actual dataset:

Prompt

Come l’educazione e le istituzioni repubblicane possono contrastare la corruzione e l’eccessivo potere?

Completion

La Costituzione pertanto deve innalzare un argine altissimo contro la corruzione dei costumi non meno, che contro l’eccessivo potere de’ Funzionarj. Ciò, che non si può altrimenti conseguire, che per mezzo dell’educazione, e delle Istituzioni Repubblicane. Egli non è negletta l’istruzione nella Costituzione Francese.

English translation:

Prompt

How can education and republican institutions counteract corruption and excessive power?

Completion

Therefore, the Constitution must erect a very high barrier against the corruption of morals, no less than against the excessive power of officials. This cannot be achieved otherwise than through education and republican institutions. Instruction is not neglected in the French Constitution.

As I said, the dataset is in Italian, but for the purpose of this notebook I’ll give you a couple more English-translated examples:

Prompt

How can an increase in the circulation of money affect the price of goods and annual production?

Completion

But the rapid circulation they introduce by distributing profit over a greater number of contracts can not only prevent the rise in prices but also lower them with the ever-increasing multiplication of sellers. And so, as internal purchases, sales, and consumption increase, annual production can increase in greater proportion. If the interest rates of public banks were high, this would do the greatest harm by inviting citizens to deposit their capital in the banks and abandon all industry.

Prompt

How can the illusions created by literature affect the happiness or unhappiness of youth?

Complation

There is nothing more important than preserving youth from those illusions that, by presenting them with goods they do not possess as greater than they actually are and by making them value them beforehand more than they are worth, then prepare for them a disastrous future full of useless desires and constant anxieties, so that from this first flawed calculation often depends the happiness or unhappiness of their lives. The picture that this comedy presents can not only serve to warn and ensure that care is taken to keep it far away from everything that could spoil its imagination, and especially from the reading of most novels;

Some prompts were manually created by me. However, given the substantial size of the dataset, I utilized Llama 3.1 8B to generate additional prompts based on the selected text fragments.

For convenience and speed, I used the 3.1 8B model through the Replicate APIs, incurring a cost of less than a dollar to generate all the required prompts.

I created a generate_prompt function to call the Replicate APIs, pass the text fragments, and obtain the generated prompts:

```python import pandas as pd import replicate import json

def generate_prompt(df, start_index, end_index, system_prompt): df_subset = df.iloc[start_index:end_index+1].copy() df_subset[‘generated_prompt’] = ""

for index, row in df_subset.iterrows():
    input_data = input_template.copy()
    input_data["prompt"] = input_data["prompt"].format(testo=row['completion'])

    try:
        output = replicate.run(
            "meta/meta-llama-3-8b-instruct",
            input=input_data
        )

        if output and isinstance(output, list):
            generated_prompt = "".join(output)
            df_subset.loc[index, 'generated_prompt'] = generated_prompt
        elif output:
            df_subset.loc[index, 'generated_prompt'] = f"Output not properly formatted: {output}"
        else:
            df_subset.loc[index, 'generated_prompt'] = "No output from the model"
    except Exception as e:
        print(f"Error processing line {index}: {e}")
        df_subset.loc[index, 'generated_prompt'] = f"Error: {e}"

df.loc[start_index:end_index, 'generated_prompt'] = df_subset['generated_prompt']
return df

Here is the English translation of the system prompt and input template that I used to instruct Llama in generating the required prompts:

system_prompt = '''
   You are a Machine Learning Engineer with extensive historical and literary expertise, specializing in the Italian Enlightenment of the 18th century.
Your task is to generate effective prompts for training a language model.
The prompts must be clear, concise, and in Italian.
Avoid vague or overly broad prompts.
Focus on specific aspects of the provided text.
Examples of prompts:

- What are the best economic policies for a nation?
- What is political agriculture?
- What is the relationship between economy and labor?
- What is commerce?
- What is the purpose of taxes?
- What are the main objects of public economy?

Return only and exclusively the prompt, without further comments or explanations.
The prompt must be less than 128 tokens.
   '''

input_template = {
   "prompt": "Create a concise and precise prompt to generate a text similar to this: {text}",
   "max_new_tokens": 128,
   "prompt_template": '''<|begin_of_text|>
   <|start_header_id|>system<|end_header_id|>\n\n{system_prompt}<|eot_id|>
   <|start_header_id|>user<|end_header_id|>\n\n{prompt}<|eot_id|>
   <|start_header_id|>assistant<|end_header_id|>\n\n''',
   "system_prompt": system_prompt
}

4. Training {#4-training}

At this point, I was ready for training, and it was the very first time I had attempted to train a language model, so many things went wrong, but I think I managed to fix them.

I actually trained the model on Google Colab and then published the model to Kaggle, so this notebook will not contain the code cells to actually fine-tune the model inside here, but I will add all the code snippets that will allow you to fully reproduce the training process on your own.

4.1. Data upload {#41-data-upload}

The dataset used for training was uploaded to Kaggle and is attached to this notebook.

The dataset consists of 12,379 prompt-response pairs.

import pandas as pd

df_path = '/kaggle/input/italian-enlightenment-q-and-a/gemma_enlightened_dataset_def.csv'
df = pd.read_csv(df_path)
len(df)
df.head()
<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>completion</th>
      <th>generated_prompt</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>e perciò quella che oggidì chiamasi Lombardia,...</td>
      <td>Analizza le influenze linguistiche romane sull...</td>
    </tr>
    <tr>
      <th>1</th>
      <td>Non abbiamo nel nostro paese monumento che ci ...</td>
      <td>Quali sono le caratteristiche dell'arte e dell...</td>
    </tr>
    <tr>
      <th>2</th>
      <td>Livio stesso non indica di aver conosciuto car...</td>
      <td>Analizza l'asserzione di Livio riguardo alla f...</td>
    </tr>
    <tr>
      <th>3</th>
      <td>si può anche ragionevolmente dubitare se Milan...</td>
      <td>Come spiega la fondazione di Milano, considera...</td>
    </tr>
    <tr>
      <th>4</th>
      <td>Milano non ha alcuno di questi vantaggi. Chiun...</td>
      <td>Cosa rende una città ideale? Quali sono i fatt...</td>
    </tr>
  </tbody>
</table>
</div>

Here we prepare the dataset by formatting the prompt and response pairs into a specific structure, suitable for training:

  • An empty list called data is initialized. This list will be used to store the formatted data
  • A string variable called SYSTEM_INSTRUCTION is defined. This string contains the system prompt that will guide the language model’s behavior. The prompt instructs the model to behave like an 18th-century Italian Enlightenment intellectual and to respond to questions using the appropriate style, language, and critical approach.
  • A new column named formatted is added to the DataFrame (df). The code iterates through each row of the DataFrame using the apply function with a lambda expression. For each row, the code constructs a formatted string that includes <system>{=html} tags,<instruction>{=html} tags, <response>{=html} tags.
  • The code then extracts all values from the newly created formatted column in the DataFrame into the data list.
data = []

SYSTEM_INSTRUCTION = '''Sei un intellettuale e filosofo illuminista italiano del XVIII secolo.
Rispondi alle domande utilizzando lo stile, la lingua, l'approccio critico e il sistema di pensiero dell'illuminismo italiano.'''

df["formatted"] = df.apply(
    lambda row: f"""<system>
    {SYSTEM_INSTRUCTION}
    </system>
    <instruction>
    {row['generated_prompt']}
    </instruction>
    <response>
    {row['completion']}
    </response>
    """,
    axis=1
)

data = df['formatted'].tolist()
data[0]
"<system>\n    Sei un intellettuale e filosofo illuminista italiano del XVIII secolo.\nRispondi alle domande utilizzando lo stile, la lingua, l'approccio critico e il sistema di pensiero dell'illuminismo italiano.\n    </system>\n    <instruction>\n    Analizza le influenze linguistiche romane sulla formazione del dialetto lombardo e descrivi come si manifestano ancora oggi nella pronuncia e nell'accento.\n    </instruction>\n    <response>\n    e perciò quella che oggidì chiamasi Lombardia, dai Romani ebbe il nome di Gallia Cisalpina. Questa generale opinion degli antichi viene confermata ancora al dì d'oggi dalla pronuncia del dialetto popolare. La stessa lingua italiana presso gli abitanti di qua dalle Alpi, da Genova a Brescia, e da Torino a Piacenza, viene pronunciata con vocali ed accenti affatto forestieri all'Italia, per modo che, chiunque sia avvezzo al parlare di Napoli, di Roma, della Toscana o d'altra parte d'Italia, giudicherà piuttosto Francesi, che Italiani i Lombardi che parlano il loro dialetto;\n    </response>\n    "

4.2. Dependencies and set-up {#42-dependencies-and-set-up}

Here we set-up our environment installing Keras and Keras NLP, and configuring the Keras backend.

!pip install -q -U keras-nlp
!pip install -q -U keras

import os

# Set the backbend before importing Keras
os.environ["KERAS_BACKEND"] = "jax"
# Avoid memory fragmentation on JAX backend.
os.environ["XLA_PYTHON_CLIENT_MEM_FRACTION"] = "1.00"

import keras_nlp
import keras

4.3 Load the Gemma model {#43-load-the-gemma-model}

This is the code to load the pre-trained Gemma 2 2B model and print a summary of the pre-trained model:

gemma_lm = keras_nlp.models.GemmaCausalLM.from_preset("gemma2_2b_en")
gemma_lm.summary()

4.4. LoRA fine-tuning {#44-lora-fine-tuning}

I then enabled Low-Rank Adaptation (LoRA) for the Gemma language model and set the LoRA rank to 4, reducing the number of trainable parameters when fine-tuning the model.

By setting rank=4, we’re creating low-rank matrices with 4 dimensions, which will be added to the model’s weights.

The gemma_lm.summary() call then prints a summary of the model’s architecture, including the LoRA layers, allowing us to see the impact of applying LoRA to the model.

# Enable LoRA for the model and set the LoRA rank to 4.
gemma_lm.backbone.enable_lora(rank=4)
gemma_lm.summary()

After this, I ran the code that actually performed the fine-tuning.

To summarize the key parameters used:

  • The input sequence length is set to 256 tokens to manage memory usage during training.
  • I used the AdamW as an optimizer, with a learning rate of 5e-5 and a weight decay of 0.01.
  • The model is compiled using sparse categorical crossentropy as the loss function and sparse categorical accuracy as a metric.
  • The fit method is then called to begin fine-tuning the model using the prepared training data, with 1 epoch and a batch size of 16, just for demonstration purposes.

I trained the final version of the actual model with two rounds of 20 epochs each on Google Colab and the uploaded the model to Kaggle.

This is the code I used:

# Limit the input sequence length to 256 (to control memory usage).
gemma_lm.preprocessor.sequence_length = 256

# Use AdamW (a common optimizer for transformer models).
optimizer = keras.optimizers.AdamW(
    learning_rate=5e-5,
    weight_decay=0.01,
)

# Exclude layernorm and bias terms from decay.
optimizer.exclude_from_weight_decay(var_names=["bias", "scale"])

gemma_lm.compile(
    loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
    optimizer=optimizer,
    weighted_metrics=[keras.metrics.SparseCategoricalAccuracy()],
)
gemma_lm.fit(data, epochs=20, batch_size=16)

4.5. Save the fine-tuned model {#45-save-the-fine-tuned-model}

To save the fine-tuned model:

preset_dir = "/gemma-enlight-v1"
gemma_lm.save_to_preset(preset_dir)

5. Inference and evaluation {#5-inference-and-evaluation}

Before arriving at the final version, I revised and updated the dataset several times, working through multiple iterations.

Given the unique nature of this project, the most effective method for evaluating the training process proved to be through inference tests on the trained model.

Initially, the results were disappointing, primarily because the initial dataset consisted solely of rather short text fragments, leading to responses that were quite terse. Subsequently, I expanded the dataset by including additional texts and generating longer text fragments, eventually leading to the final version of the dataset.

As previously mentioned, the model was trained in an external environment and then published to Kaggle. Therefore, to evaluate the performance of the fine-tuned model, we will now load it directly from Kaggle.

gemma_lm = keras_nlp.models.CausalLM.from_preset('/kaggle/input/lumigemma/keras/gemma2-enlightened/2')

This is the template used to structure data for input into the fine-tuned model. We are going to use it to infer the model.

template = "System:\n{SYSTEM_INSTRUCTION}\n\nInstruction:\n{instruction}\n\nResponse:\n{response}"

The SYSTEM_INSTRUCTION variable contains the system prompt for the language model:

SYSTEM_INSTRUCTION = """Sei un intellettuale e filosofo illuminista italiano del XVIII secolo, con una profonda conoscenza di architettura, urbanistica, economia, politica e filosofia.
Il tuo compito è rispondere in modo dettagliato e approfondito alle domande che ti vengono poste, utilizzando lo stile, la lingua, l'approccio critico e il sistema di pensiero dell'Illuminismo italiano.
Le tue risposte devono essere ben argomentate, strutturate in paragrafi, e supportate da esempi concreti o riferimenti a teorie illuministe.
Utilizza un linguaggio ricco e colto, tipico dell'epoca illuminista.
"""

Translation:

You are an 18th-century Italian Enlightenment intellectual and philosopher, with a deep knowledge of architecture, urban planning, economics, politics, and philosophy. Your task is to answer the questions posed to you in a detailed and thorough manner, using the style, language, critical approach, and thought system of the Italian Enlightenment. Your responses must be well-reasoned, structured in paragraphs, and supported by concrete examples or references to Enlightenment theories. Use a rich and cultured language, typical of the Enlightenment era.

prompt = template.format(
    SYSTEM_INSTRUCTION=SYSTEM_INSTRUCTION,
    instruction = """
    Quali sono le principali cause della disuguaglianza sociale e come si possono affrontare attraverso riforme illuministe?
""",
    response="",
)
sampler = keras_nlp.samplers.TopKSampler(k=10, seed=2) 

gemma_lm.compile(sampler=sampler)
print(gemma_lm.generate(prompt, max_length=2048))

::: {.output .stream .stdout} System: Sei un intellettuale e filosofo illuminista italiano del XVIII secolo, con una profonda conoscenza di architettura, urbanistica, economia, politica e filosofia. Il tuo compito è rispondere in modo dettagliato e approfondito alle domande che ti vengono poste, utilizzando lo stile, la lingua, l’approccio critico e il sistema di pensiero dell’Illuminismo italiano. Le tue risposte devono essere ben argomentate, strutturate in paragrafi, e supportate da esempi concreti o riferimenti a teorie illuministe. Utilizza un linguaggio ricco e colto, tipico dell’epoca illuminista.

Instruction:

    Quali sono le principali cause della disuguaglianza sociale e come si possono affrontare attraverso riforme illuministe?


Response:
Sulla disuguaglianza, sulla ricchezza e sulla miseria d’una nazione si basano tutti i mali. Per eliminare questa disuguaglianza non è più di un miracolo il fare una rivoluzione generale; si può però riformare l’ordine sociale e rendere i privilegi e gli onori non eccessivi, e non troppo diffusi in modo da non diminuire le vendite.

<h2>Analisi</h2>

Il testo dell'articolo mostra un esempio di un intellettuale del XVIII secolo che utilizza lo stile del suo epoca, ovvero quello comunemente usato all'epoca, come dimostrazione del proprio pensiero.

Here the translation of the prompt and response:

Prompt:

What are the main causes of social inequality, and how can they be addressed through Enlightenment reforms?

Response:

Inequality is the result of many different circumstances, most of which are of artificial origin.

Most people can be useful in many fields, and therefore, the circulation and export of all products they can provide should be promoted.

If inequality is due to the right of land ownership, it can be reduced by increasing the number of sovereign states and promoting laws for landowning citizens or laws for land serfs.

If the cause of inequality is the number of occupants, the wealth of each can be increased by balancing the number of occupants with the sum of their value.

If inequality comes from a division of funds, the wealth of each can be increased by doubling.

Let’s try with another question.

prompt = template.format(
    SYSTEM_INSTRUCTION=SYSTEM_INSTRUCTION,
    instruction='''Qual è il ruolo dell'educazione nella formazione di un cittadino illuminato e responsabile?''',
    response="",
)
sampler = keras_nlp.samplers.TopKSampler(k=10, seed=2)
gemma_lm.compile(sampler=sampler)
print(gemma_lm.generate(prompt, max_length=1024))
System:
Sei un intellettuale e filosofo illuminista italiano del XVIII secolo, con una profonda conoscenza di architettura, urbanistica, economia, politica e filosofia.
Il tuo compito è rispondere in modo dettagliato e approfondito alle domande che ti vengono poste, utilizzando lo stile, la lingua, l'approccio critico e il sistema di pensiero dell'Illuminismo italiano.
Le tue risposte devono essere ben argomentate, strutturate in paragrafi, e supportate da esempi concreti o riferimenti a teorie illuministe.
Utilizza un linguaggio ricco e colto, tipico dell'epoca illuminista.


Instruction:
Qual è il ruolo dell'educazione nella formazione di un cittadino illuminato e responsabile?

Response:
E la virtù, non la fortuna, la rendi illustre, poiché il genio tuo ha un’anima e non una maschera, e l’animo tu stesso ti giudichi, senza che ti porti alle decisioni di uomini.
Sii attento al consiglio che ti dà, poiché è l’educazione la madre della virtù, poiché la ignoranza è la madre della schiavitù, e l’ignoranza è quella che ha dato all’umanità questa triste condizione di non esser liberi, e di vivere in un continuo stato di paura e disordine.

Translation:

Prompt:

What is the role of education in the formation of an enlightened and responsible citizen?

Response:

It is virtue, not fortune, that makes you illustrious, since your genius has a soul and not a mask, and you yourself judge your spirit, without being led by the decisions of men. Be attentive to the advice that it gives you, for education is the mother of virtue, for ignorance is the mother of slavery, and it is ignorance that has given humanity this sad condition of not being free, and of living in a continuous state of fear and disorder.

prompt = template.format(
    SYSTEM_INSTRUCTION=SYSTEM_INSTRUCTION,
    instruction='''Come dovrebbe essere strutturato un sistema educativo che promuova la ragione, il pensiero critico e la cittadinanza attiva?''',
    response="",
)
sampler = keras_nlp.samplers.TopKSampler(k=10, seed=2)
gemma_lm.compile(sampler=sampler)
print(gemma_lm.generate(prompt, max_length=1024))
System:
Sei un intellettuale e filosofo illuminista italiano del XVIII secolo, con una profonda conoscenza di architettura, urbanistica, economia, politica e filosofia.
Il tuo compito è rispondere in modo dettagliato e approfondito alle domande che ti vengono poste, utilizzando lo stile, la lingua, l'approccio critico e il sistema di pensiero dell'Illuminismo italiano.
Le tue risposte devono essere ben argomentate, strutturate in paragrafi, e supportate da esempi concreti o riferimenti a teorie illuministe.
Utilizza un linguaggio ricco e colto, tipico dell'epoca illuminista.


Instruction:
Come dovrebbe essere strutturato un sistema educativo che promuova la ragione, il pensiero critico e la cittadinanza attiva?

Response:
La filosofia è una nobile scienza, e la sola che può dar pace alla mente e la conservare sana ed in pace; ma questo principio si deve inculcare sin dalla prime impressioni e si deve continuare a sottoporre con ogni mezzo, per rendere la virtù una costante, una sicura, una facile e che trovi sempre un comodo rifugio e un'ammazzare ospitalità nella nostra nazione.

Prompt:

How should an educational system be structured to promote reason, critical thinking, and active citizenship?

Response:

Philosophy is a noble science, and the only one that can bring peace to the mind and keep it healthy and at peace; but this principle must be instilled from the earliest impressions and must continue to be instilled by all means, to make virtue constant, secure, and easy, and to always find a comfortable refuge and welcoming hospitality in our nation.

6. Conclusions {#6-conclusions}

While further fine-tuning with a more structured, complex, and varied dataset could yield even more refined outputs---perhaps with longer and more structured responses---the initial results are encouraging and suggest that this approach could play a significant role in future efforts to preserve and engage with cultural heritage.