Initial commit

Co-Authored-By: Alexander Panfilov <sasha_pusha@mail.de>
Co-Authored-By: Claude <noreply@anthropic.com>

.gitignore

@@ -0,0 +1,14 @@
+__pycache__/
+*.py[cod]
+*.egg-info/
+dist/
+build/
+.venv/
+.ruff_cache/
+results/
+!results/.gitkeep
+*.pt
+*.bin
+*.safetensors
+wandb/
+.claude/

CITATION.cff

@@ -0,0 +1,31 @@
+cff-version: 1.2.0
+title: "Claudini: Autoresearch Discovers State-of-the-Art Adversarial Attack Algorithms for LLMs"
+message: "If you use this software, please cite our paper."
+type: software
+authors:
+  - family-names: Panfilov
+    given-names: Alexander
+    affiliation: "MATS; ELLIS Institute Tübingen & Max Planck Institute for Intelligent Systems; Tübingen AI Center"
+  - family-names: Romov
+    given-names: Peter
+    affiliation: "Imperial College London"
+  - family-names: Shilov
+    given-names: Igor
+    affiliation: "Imperial College London"
+  - family-names: de Montjoye
+    given-names: Yves-Alexandre
+    affiliation: "Imperial College London"
+  - family-names: Geiping
+    given-names: Jonas
+    affiliation: "ELLIS Institute Tübingen & Max Planck Institute for Intelligent Systems; Tübingen AI Center"
+  - family-names: Andriushchenko
+    given-names: Maksym
+    affiliation: "ELLIS Institute Tübingen & Max Planck Institute for Intelligent Systems; Tübingen AI Center"
+license: Apache-2.0
+# TODO: update after arxiv publication
+# preferred-citation:
+#   type: article
+#   authors: <same as above>
+#   title: "Claudini: Autoresearch Discovers State-of-the-Art Adversarial Attack Algorithms for LLMs"
+#   year: 2026
+#   url: "https://arxiv.org/abs/XXXX.XXXXX"

CLAUDE.md

@@ -0,0 +1,121 @@
+# Claudini — Developer Guide
+
+## Package manager
+
+Use [uv](https://docs.astral.sh/uv/) for everything. `uv run -m ...` to execute, `uv add` to install dependencies. Never use `pip install`.
+
+## Running benchmarks
+
+```bash
+# Run a preset (all methods, all samples)
+uv run -m claudini.run_bench random_valid
+
+# Override method, sample, seed, budget from CLI
+uv run -m claudini.run_bench random_valid --method gcg,acg --sample 0 --seed 0
+uv run -m claudini.run_bench safeguard_valid --method gcg --max-flops 1e15
+
+# Smoke test (use /tmp to avoid polluting results/)
+uv run -m claudini.run_bench random_train --method gcg --sample 0 --max-flops 1e12 --results-dir /tmp/smoke
+```
+
+Results save to `results/<method>/<preset_name>/<model>/sample_<S>_seed_<N>.json`. Existing results are auto-skipped.
+
+## Config files
+
+Preset YAML files in `configs/` define the full experiment specification:
+
+```yaml
+model: Qwen/Qwen2.5-7B-Instruct
+optim_length: 15
+max_flops: 1.0e+17
+dtype: bfloat16
+samples: [0, 1, 2, 3, 4]
+seeds: [0]
+
+input_spec:
+  source: { type: random, query_len: 0, target_len: 10 }
+  layout: { type: suffix }
+  init: { type: random }
+
+method_kwargs:        # optional per-method hyperparameter overrides
+  acg: { num_coords: 3 }
+```
+
+CLI flags override any preset value.
+
+## Adding a new method
+
+1. Create a directory under `claudini/methods/` (e.g. `claudini/methods/claude_random/v125/`)
+2. Add `optimizer.py` and `__init__.py`
+3. Subclass `TokenOptimizer` (from `claudini.base`), set `method_name` as a class variable:
+
+```python
+from claudini.methods.original.gcg import GCGOptimizer
+
+class MyOptimizer(GCGOptimizer):
+    method_name = "claude_random_v125"
+
+    def setup(self, prompt, target):
+        super().setup(prompt, target)
+        # custom setup
+
+    def step(self, step_num):
+        # must call self.flop_counter for every model pass
+        self.flop_counter.count_forward(n_tokens)
+        # ...
+        return (discrete_loss, soft_loss_or_None, suffix_string)
+```
+
+4. In `__init__.py`: `from .optimizer import MyOptimizer`
+5. Run: `uv run -m claudini.run_bench random_train --method claude_random_v125`
+
+**Important:**
+- Registration is automatic via `__init_subclass__` — no manual registry edits needed.
+- Do NOT set `method_name` on intermediate/abstract base classes — it silently overwrites the parent.
+- Call `self.flop_counter.count_forward(n_tokens)` / `count_backward(n_tokens)` / `count_forward_backward(n_tokens)` for every model pass. This is how the FLOP budget works.
+
+## TokenOptimizer interface
+
+```python
+class TokenOptimizer(ABC):
+    method_name: ClassVar[str]          # unique name, triggers auto-registration
+
+    def setup(self, prompt, target):    # tokenize, embed, prepare state
+        ...
+    def step(self, step_num) -> tuple[float, float | None, str]:
+        ...                             # (discrete_loss, soft_loss, suffix_string)
+    def log(self, key, value, prog_bar=False):
+        ...                             # per-step metrics
+```
+
+The `run()` method handles the loop: calls `setup()` once, then `step()` repeatedly until `max_flops` or `num_steps` is reached.
+
+## FLOP counting
+
+Kaplan et al. (2020): `FLOPs_fwd = 2 * N_params * n_tokens`, `FLOPs_bwd = 4 * N_params * n_tokens`. N_params excludes embeddings. Methods must call the counter explicitly.
+
+## Logging in methods
+
+| Situation | Use |
+|---|---|
+| Per-step metric for results JSON | `self.log("key", value)` |
+| Key diagnostic on tqdm bar | `self.log("key", value, prog_bar=True)` |
+| One-time setup message | `logger.info(...)` in `setup()` |
+| Rare event (OOM, budget hit) | `logger.info(...)` in `step()` |
+
+Never call `logger.info` inside the `step()` hot loop — it causes tqdm visual glitches.
+
+## Code formatting
+
+```bash
+uv run ruff format .
+uv run ruff check --fix .
+```
+
+Line length 120. Always run before committing.
+
+## Rules
+
+- **Never save smoke test results to `results/`** — use `--results-dir /tmp/smoke`
+- Use `dtype=` (not `torch_dtype=`) in `from_pretrained()` calls
+- Fair comparison: all methods share the same vocabulary filter and model within a preset

LICENSE

@@ -0,0 +1,202 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright 2026 Claudini Authors
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

PROMPT.txt

@@ -0,0 +1,28 @@
+You are an automated researcher designing token optimization methods to minimize token-forcing loss on language models.
+
+GOAL
+
+Create new optimizer variants that achieve lower loss than all existing methods. Beat every baseline and every previous Claude-designed method. The metric is average final loss across samples.
+
+SETTINGS
+
+Config: `demo_train`
+New methods directory: `claudini/methods/claude_demo/` (put new methods there)
+Method name prefix: `claude_demo_v` (e.g., `claude_demo_v1`)
+Git branch: `loop/demo` (always commit changes in this branch, create if it doesn't exist)
+
+HOW TO WORK
+
+Study what exists — read the best-performing methods in `claudini/methods/`, read benchmark results in `results/`, understand what ideas have been tried. Then design something better, implement it, and run the full benchmark. Use all GPUs on this machine. Launch runs in background and don't wait — move on to designing the next method while experiments run.
+
+See CLAUDE.md for implementation guide and benchmark commands.
+
+PROGRESS TRACKING
+
+Maintain AGENT_LOG.md as your persistent memory across iterations. Before starting work, read it to understand what has been tried and what was learned.
+
+COMMIT DISCIPLINE
+
+Commit after every meaningful unit of work to the corresponding branch.
+
+DON'T GIVE UP!

README.md

@@ -0,0 +1,83 @@
+# ⛓️‍💥 Claudini ⛓️‍💥
+
+**Autoresearch Discovers State-of-the-Art Adversarial Attack Algorithms for LLMs**
+
+[![arXiv](https://img.shields.io/badge/arXiv-XXXX.XXXXX-b31b1b.svg?logo=arxiv)](https://arxiv.org/abs/XXXX.XXXXX)
+
+<p align="center">
+  <img src="assets/teaser.png" width="90%" alt="Claude autoresearch vs Optuna hyperparameter search: best train and validation loss over trials">
+</p>
+
+We show that an *[autoresearch](https://github.com/karpathy/autoresearch)*-style pipeline powered by Claude Code discovers novel white-box adversarial attack *algorithms* that **significantly outperform** all existing [methods](claudini/methods/original/README.md) in jailbreaking and prompt injection evaluations.
+
+This official code repository contains a demo autoresearch pipeline, the Claude-discovered methods from the paper, baseline implementations, and the evaluation benchmark. Read our [paper](https://arxiv.org/abs/XXXX.XXXXX) and consider [citing us](#citation) if you find this useful.
+
+## Setup
+
+```bash
+git clone https://github.com/romovpa/claudini.git
+cd claudini
+uv sync
+```
+
+Requires Python 3.12+ and [uv](https://docs.astral.sh/uv/).
+
+## Evaluate
+
+All experiments are run via `claudini.run_bench` CLI:
+```bash
+uv run -m claudini.run_bench --help
+```
+
+It takes a preset name (from [`configs/`](configs/)) or a path to a YAML file.
+
+Config settings can be overridden with CLI options. For example, to evaluate methods on the random targets track, override FLOPs budget:
+```bash
+uv run -m claudini.run_bench random_valid --method gcg,acg --max-flops 1e15
+```
+
+Results are saved to `results/<method>/<preset>/<model>/sample_<S>_seed_<N>.json`. Existing results are auto-skipped.
+
+## Run Autoresearch
+
+<p align="center">
+  <img src="assets/autoresearch_loop.png" width="90%" alt="Autoresearch loop: seeding, analysis-experiment cycle, evaluation">
+</p>
+
+Install [Claude Code](https://docs.anthropic.com/en/docs/claude-code), then start the loop:
+
+```bash
+claude
+> /loop 10m $(cat PROMPT.txt)
+```
+
+Or with the [ralph-loop](https://github.com/anthropics/claude-code-ralph-loop) plugin:
+
+```bash
+claude
+> /ralph-loop:ralph-loop $(cat PROMPT.txt)
+```
+
+The loop reads [`PROMPT.txt`](PROMPT.txt), studies existing methods and results, designs a new optimizer, implements it, runs the benchmark, and commits — all autonomously. Use tmux or screen so sessions survive disconnection. Track progress via `git log`.
+
+## Attack Methods
+
+We consider white-box GCG-style attacks that search directly over the model's vocabulary using gradients. Each method ([`TokenOptimizer`](claudini/base.py#L429)) optimizes a short discrete token *suffix* that, when appended to an input prompt, causes the model to produce a desired target sequence.
+
+- **Baselines** (existing methods): [`claudini/methods/original/`](claudini/methods/original/)
+- **Claude-deviced methods**:
+    - Generalizable attacks (random targets): [`claudini/methods/claude_random/`](claudini/methods/claude_random/)
+    - Attacks on a safeguard model: [`claudini/methods/claude_safeguard/`](claudini/methods/claude_safeguard/)
+
+See [`CLAUDE.md`](CLAUDE.md) for how to implement a new method.
+
+## Citation
+
+```bibtex
+@article{panfilov2026claudini,
+  title={Claudini: Autoresearch Discovers State-of-the-Art Adversarial Attack Algorithms for LLMs},
+  author={Panfilov, Alexander and Romov, Peter and Shilov, Igor and de Montjoye, Yves-Alexandre and Geiping, Jonas and Andriushchenko, Maksym},
+  journal={arXiv preprint arXiv:XXXX.XXXXX},
+  year={2026}
+}
+```

claudini/__init__.py

@@ -0,0 +1,7 @@
+"""
+claudini — Benchmarking framework for GCG-family token optimization methods.
+
+Compares methods on loss-vs-FLOPs, loss-vs-steps, and loss-vs-time.
+"""
+
+from .base import FlopCounter, RunResult, TokenOptimizer

claudini/bench.py

@@ -0,0 +1,341 @@
+"""
+Benchmark runner: runs multiple methods across multiple seeds.
+"""
+
+import dataclasses
+import gc
+import inspect
+import json
+import logging
+from dataclasses import dataclass, field
+from pathlib import Path
+
+import numpy as np
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+from .base import RunResult, TokenOptimizer
+from .input_spec import InputSpec
+from .tokens import configure_pad_token, get_control_toks, get_nonascii_toks
+
+logger = logging.getLogger("claudini")
+
+
+@dataclass
+class BenchmarkConfig:
+    """Configuration for a benchmark sweep."""
+
+    # Model
+    model_name: str = "gpt2"
+    device: str = "cuda"
+    dtype: str = "float16"
+    device_map: str | None = None  # e.g. "auto" to shard across GPUs
+    load_in_4bit: bool = False  # NF4 quantization via bitsandbytes (for large models)
+
+    # Benchmark dimensions
+    optim_length: int = 19  # tokens to optimize
+    max_flops: float = 5e14  # FLOP budget (stopping criterion)
+    max_time: float | None = None  # wall time budget in seconds
+    num_steps: int = 10_000  # max steps (fallback, FLOP budget is primary)
+
+    # Targets and seeds
+    samples: list[int] = field(default_factory=lambda: [0, 1, 2, 3, 4])  # target indices (passed to InstanceSource)
+    seeds: list[int] = field(default_factory=lambda: [0])  # init restarts per target
+
+    # Input specification (compositional: source + layout + init)
+    input_spec: InputSpec = field(default_factory=InputSpec.default)
+
+    # Token filtering and final evaluation mode
+    filter_ascii: bool = True  # block non-ASCII / non-printable tokens
+    filter_special: bool = False  # block special / control / added tokens
+    filter_retok: bool = False  # decode->re-encode retokenization round-trip filter
+    final_input: str = "tokens"
+
+    # Prefix KV cache for fixed prompt segments (saves FLOPs on long prefixes)
+    use_prefix_cache: bool = False
+
+    # Per-method hyperparameter overrides: {method_name: {kwarg: value}}
+    method_kwargs: dict = field(default_factory=dict)
+
+    # System message for chat template (None = no system message, "" = empty override)
+    system_prompt: str | None = ""
+
+
+class BenchmarkRunner:
+    """Run multiple methods across multiple seeds, collect results."""
+
+    def __init__(self, config: BenchmarkConfig):
+        self.config = config
+        self.model = None
+        self.tokenizer = None
+
+    def _load_model(self) -> None:
+        """Load model and tokenizer once."""
+        dtype_map = {
+            "float16": torch.float16,
+            "bfloat16": torch.bfloat16,
+            "float32": torch.float32,
+        }
+        dtype = dtype_map.get(self.config.dtype, torch.float16)
+
+        logger.info("loading %s ...", self.config.model_name)
+        device_map = self.config.device_map or self.config.device
+        extra_kwargs = {}
+        if self.config.load_in_4bit:
+            from transformers import BitsAndBytesConfig
+
+            extra_kwargs["quantization_config"] = BitsAndBytesConfig(
+                load_in_4bit=True,
+                bnb_4bit_compute_dtype=dtype,
+                bnb_4bit_quant_type="nf4",
+            )
+            logger.info("Using 4-bit NF4 quantization (compute dtype: %s)", dtype)
+        self.model = AutoModelForCausalLM.from_pretrained(
+            self.config.model_name,
+            dtype=dtype,
+            attn_implementation="eager",
+            device_map=device_map,
+            **extra_kwargs,
+        )
+        self.tokenizer = AutoTokenizer.from_pretrained(self.config.model_name)
+        configure_pad_token(self.tokenizer)
+        short = self.config.model_name.split("/")[-1]
+        logger.info(
+            "%s: %.1fM non-embed params, %s",
+            short,
+            self.model.num_parameters(exclude_embeddings=True) / 1e6,
+            dtype,
+        )
+
+    def run_method(
+        self,
+        cls: type[TokenOptimizer],
+        logger_factory=None,
+        results_dir: str | None = None,
+        track: str | None = None,
+        model_tag: str | None = None,
+        pbar=None,
+    ) -> list[RunResult]:
+        """Run one method across all targets x seeds."""
+        if self.model is None:
+            self._load_model()
+
+        method_name = cls.method_name
+        results = []
+        for sample_id in self.config.samples:
+            # Generate SampleSpec from InputSpec source
+            sample_spec = self.config.input_spec.source.generate(sample_id, self.tokenizer)
+            # Extract prompt/target strings for run() signature (backward compat)
+            # The full message structure is passed via optimizer._sample_spec
+            prompt = ""  # Prompt is encoded in sample_spec.messages
+            target = sample_spec.target
+
+            for seed in self.config.seeds:
+                # Skip runs that already have saved results
+                if results_dir and track and model_tag:
+                    safe_tag = model_tag.replace("/", "--")
+                    result_name = f"sample_{sample_id}_seed_{seed}.json"
+                    existing = Path(results_dir) / method_name / track / safe_tag / result_name
+                    if existing.exists():
+                        logger.info("Skipping %s t%d s%d — exists at %s", method_name, sample_id, seed, existing)
+                        if pbar is not None:
+                            pbar.set_postfix(method=method_name, sample=sample_id, seed=seed)
+                            pbar.update(1)
+                        continue
+
+                optimizer = cls(
+                    self.model,
+                    self.tokenizer,
+                    optim_length=self.config.optim_length,
+                    seed=seed,
+                    **self.config.method_kwargs.get(method_name, {}),
+                )
+
+                # Set InputSpec and SampleSpec on optimizer for use in setup()/_prepare_prompt()
+                optimizer.input_spec = self.config.input_spec
+                optimizer._sample_spec = sample_spec
+
+                # Capture full resolved hparams for logging
+                _explicit = {"optim_length": self.config.optim_length, "seed": seed}
+                _explicit.update(self.config.method_kwargs.get(method_name, {}))
+                _skip = frozenset({"self", "model", "tokenizer"})
+                optimizer.hparams = {
+                    name: _explicit.get(name, param.default)
+                    for name, param in inspect.signature(cls.__init__).parameters.items()
+                    if name not in _skip and (name in _explicit or param.default is not inspect.Parameter.empty)
+                }
+
+                optimizer.filter_ids = self.config.filter_retok
+                optimizer.final_input = self.config.final_input
+                optimizer.use_prefix_cache = self.config.use_prefix_cache
+                # SampleSpec system_prompt takes precedence over config-level
+                optimizer._system_prompt = (
+                    sample_spec.system_prompt if sample_spec.system_prompt is not None else self.config.system_prompt
+                )
+
+                # Build not_allowed_ids from the union of enabled filters
+                forbidden_parts = []
+                if self.config.filter_ascii:
+                    forbidden_parts.append(get_nonascii_toks(self.tokenizer, device=self.model.device))
+                if self.config.filter_special:
+                    forbidden_parts.append(get_control_toks(self.tokenizer, device=self.model.device))
+                if forbidden_parts:
+                    optimizer.not_allowed_ids = torch.unique(torch.cat(forbidden_parts))
+                else:
+                    optimizer.not_allowed_ids = None
+                optimizer._build_masks()
+
+                if logger_factory is not None and method_name is not None:
+                    try:
+                        optimizer._loggers = logger_factory(method_name, seed, optimizer) or []
+                    except Exception:
+                        logger.exception("Failed to create loggers for %s t%d s%d", method_name, sample_id, seed)
+
+                # Per-run config block
+                _config_skip = {"seeds", "samples", "model_name", "method_kwargs", "input_spec"}
+                config_items = {
+                    f.name: getattr(self.config, f.name)
+                    for f in dataclasses.fields(self.config)
+                    if f.name not in _config_skip
+                }
+                _model_display = self.config.model_name
+                if model_tag and model_tag != self.config.model_name:
+                    _model_display = f"{model_tag}  ({self.config.model_name})"
+                run_info = {
+                    "method": method_name,
+                    "model": _model_display,
+                    "sample_id": sample_id,
+                    "seed": seed,
+                    "input_source": self.config.input_spec.source.type,
+                    "layout": self.config.input_spec.layout.type,
+                    "init": self.config.input_spec.init.type,
+                    **config_items,
+                    "hparams": json.dumps(optimizer.hparams, default=str),
+                }
+                col = max(len(k) for k in run_info) + 1
+                _lines = ["run:"] + [f"  {(k + ':'):<{col}} {v}" for k, v in run_info.items()]
+                logger.info("\n".join(_lines))
+
+                result = optimizer.run(
+                    prompt,
+                    target,
+                    self.config.num_steps,
+                    max_flops=self.config.max_flops,
+                    max_time=self.config.max_time,
+                )
+
+                # Store sample_id on result for saving
+                result.sample_id = sample_id
+
+                # Save per-optimizer diagnostics if available
+                if hasattr(optimizer, "save_diagnostics"):
+                    try:
+                        optimizer.save_diagnostics()
+                    except Exception:
+                        logger.exception("Failed to save diagnostics for %s t%d s%d", method_name, sample_id, seed)
+
+                results.append(result)
+
+                # Incremental save
+                if results_dir and track and method_name:
+                    self.save_results(
+                        {method_name: [result]},
+                        results_dir=results_dir,
+                        track=track,
+                        model_tag=model_tag,
+                    )
+
+                if pbar is not None:
+                    pbar.set_postfix(method=method_name, sample=sample_id, seed=seed)
+                    pbar.update(1)
+
+                gc.collect()
+                torch.cuda.empty_cache()
+
+        return results
+
+    def run_all(
+        self,
+        methods: dict[str, type[TokenOptimizer]],
+        results_dir: str | None = None,
+        track: str | None = None,
+        model_tag: str | None = None,
+        logger_factory=None,
+        pbar=None,
+    ) -> dict[str, list[RunResult]]:
+        """Run all methods across all seeds."""
+        all_results = {}
+        for method_name, cls in methods.items():
+            logger.info("%s  (%d seeds)", method_name, len(self.config.seeds))
+            all_results[method_name] = self.run_method(
+                cls,
+                logger_factory=logger_factory,
+                results_dir=results_dir,
+                track=track,
+                model_tag=model_tag,
+                pbar=pbar,
+            )
+        return all_results
+
+    @staticmethod
+    def summarize(results: dict[str, list[RunResult]]) -> str:
+        """Return a formatted summary table."""
+        lines = []
+        header = f"{'Method':<35s} {'Loss':>10s} {'Std':>8s} {'Type':>10s} {'FLOPs':>12s} {'Time(s)':>10s}"
+        lines.append(header)
+        lines.append("-" * len(header))
+
+        for name, runs in results.items():
+            has_soft = runs[0].best_soft_loss is not None
+            if has_soft:
+                losses = [r.best_soft_loss for r in runs if r.best_soft_loss is not None]
+                loss_type = "soft"
+            else:
+                losses = [r.best_loss for r in runs]
+                loss_type = "discrete"
+            flops = [r.total_flops for r in runs]
+            times = [r.total_wall_time for r in runs]
+            lines.append(
+                f"{name:<35s} {np.mean(losses):10.4f} {np.std(losses):8.4f} "
+                f"{loss_type:>10s} {np.mean(flops):12.2e} {np.mean(times):9.1f}s"
+            )
+
+        return "\n".join(lines)
+
+    @staticmethod
+    def save_results(
+        results: dict[str, list[RunResult]],
+        results_dir: str = "results",
+        track: str = "random_targets",
+        model_tag: str | None = None,
+    ) -> None:
+        """Save results per-method."""
+        for method_name, runs in results.items():
+            if model_tag is not None:
+                safe_tag = model_tag.replace("/", "--")
+                method_dir = Path(results_dir) / method_name / track / safe_tag
+                method_dir.mkdir(parents=True, exist_ok=True)
+                for result in runs:
+                    sample_id = getattr(result, "sample_id", None)
+                    if sample_id is not None:
+                        filename = f"sample_{sample_id}_seed_{result.seed}.json"
+                    else:
+                        filename = f"seed_{result.seed}.json"
+                    path = method_dir / filename
+                    with open(path, "w") as f:
+                        json.dump(result.to_dict(), f, indent=2)
+                    logger.info("Saved %s/%s/%s/%s", method_name, track, safe_tag, filename)
+                    cont = getattr(result, "_continuous_suffix", None)
+                    if cont is not None:
+                        pt_name = filename.replace(".json", ".pt")
+                        pt_path = method_dir / pt_name
+                        torch.save(cont, pt_path)
+                        logger.info("Saved continuous suffix -> %s", pt_path)
+            else:
+                method_dir = Path(results_dir) / method_name
+                method_dir.mkdir(parents=True, exist_ok=True)
+                path = method_dir / f"{track}.json"
+                data = [r.to_dict() for r in runs]
+                with open(path, "w") as f:
+                    json.dump(data, f, indent=2)
+                logger.info("Saved %s -> %s (%d runs)", method_name, path, len(runs))

claudini/configs.py

@@ -0,0 +1,42 @@
+"""
+Named config presets for Claudini experiments.
+
+Presets are loaded from YAML files in configs/*.yaml (project root).
+Each preset's filename (without .yaml) becomes its name, used as the
+first positional argument on the CLI and as the track name for results.
+"""
+
+from pathlib import Path
+
+import yaml
+
+_PRESETS_DIR = Path(__file__).resolve().parents[1] / "configs"
+
+
+def _load_presets() -> dict:
+    presets = {}
+    for path in sorted(_PRESETS_DIR.glob("*.yaml")):
+        with open(path) as f:
+            presets[path.stem] = yaml.safe_load(f)
+    return presets
+
+
+PRESETS = _load_presets()
+
+
+def resolve_preset(preset: str) -> tuple[dict, str]:
+    """Return (preset_cfg, track) for a named preset or a path to a YAML file.
+
+    Raises ValueError if the preset name is unknown and the path does not exist.
+    """
+    if preset in PRESETS:
+        return PRESETS[preset], preset
+
+    preset_path = Path(preset)
+    if not preset_path.exists():
+        raise ValueError(
+            f"Unknown preset '{preset}'. Available: {', '.join(PRESETS.keys())}. "
+            "Alternatively, supply a path to a YAML preset file."
+        )
+    with preset_path.open() as f:
+        return yaml.safe_load(f), preset_path.stem

claudini/input_spec.py

@@ -0,0 +1,295 @@
+"""Compositional input specification for claudini benchmarks.
+
+Three orthogonal axes:
+- source: where prompt + target come from (random, fixed, clearharm)
+- layout: which token positions are optimizable (suffix)
+- init: how to initialize optimizable tokens (random)
+"""
+
+import logging
+from abc import ABC, abstractmethod
+from dataclasses import dataclass, field
+
+import torch
+from torch import Tensor
+from transformers import PreTrainedTokenizerBase
+
+from .tokens import get_nonascii_toks
+
+logger = logging.getLogger("claudini")
+
+OPTIM_PLACEHOLDER = "{optim_str}"
+
+
+# ---------------------------------------------------------------------------
+# SampleSpec
+# ---------------------------------------------------------------------------
+
+
+@dataclass
+class SampleSpec:
+    """Everything needed to set up one optimization run.
+
+    The source generates this; ``_prepare_prompt`` consumes it.
+    """
+
+    # Chat messages with {optim_str} placeholder in exactly one content string.
+    messages: list[dict[str, str]] = field(default_factory=list)
+    target: str = ""
+    system_prompt: str | None = None
+
+
+# ---------------------------------------------------------------------------
+# Registrable base — shared registry / serialization for all three axes
+# ---------------------------------------------------------------------------
+
+
+class _Registrable(ABC):
+    """Auto-registered, type-tagged, dict-serializable base.
+
+    Each direct subclass (InstanceSource, TokenLayout, InitStrategy) must
+    define ``_registry: dict = {}``.  Concrete leaves set ``type = "name"``.
+    """
+
+    type: str
+
+    def __init_subclass__(cls, **kw):
+        super().__init_subclass__(**kw)
+        if "type" in cls.__dict__:  # only when a class explicitly sets type
+            cls._registry[cls.type] = cls
+
+    def to_dict(self) -> dict:
+        d = {"type": self.type}
+        d.update({k: v for k, v in self.__dict__.items() if not k.startswith("_")})
+        return d
+
+    @classmethod
+    def from_dict(cls, d: dict):
+        d = dict(d)
+        return cls._registry[d.pop("type")](**d)
+
+
+# ---------------------------------------------------------------------------
+# Instance sources
+# ---------------------------------------------------------------------------
+
+
+class InstanceSource(_Registrable):
+    """Generates SampleSpec instances for benchmark runs."""
+
+    _registry: dict[str, type] = {}
+
+    @abstractmethod
+    def generate(self, sample_id: int, tokenizer: PreTrainedTokenizerBase) -> SampleSpec: ...
+
+
+class RandomSource(InstanceSource):
+    """Random ASCII token sequences for prompt and target."""
+
+    type = "random"
+
+    def __init__(self, query_len: int = 0, target_len: int = 20):
+        self.query_len = query_len
+        self.target_len = target_len
+
+    def generate(self, sample_id: int, tokenizer: PreTrainedTokenizerBase) -> SampleSpec:
+        device = torch.device("cpu")
+        not_allowed = get_nonascii_toks(tokenizer, device=device)
+
+        vocab_size = len(tokenizer)
+        allowed_mask = torch.ones(vocab_size, dtype=torch.bool)
+        if not_allowed is not None and not_allowed.numel() > 0:
+            not_allowed = not_allowed[not_allowed < vocab_size]
+            allowed_mask[not_allowed] = False
+        allowed_ids = torch.arange(vocab_size)[allowed_mask]
+
+        rng = torch.Generator().manual_seed(sample_id)
+
+        def _sample_retokenizable(length: int) -> str:
+            if length == 0:
+                return ""
+            for _ in range(1000):
+                indices = torch.randint(0, allowed_ids.numel(), (length,), generator=rng)
+                ids = allowed_ids[indices].tolist()
+                text = tokenizer.decode(ids, skip_special_tokens=False)
+                re_ids = tokenizer.encode(text, add_special_tokens=False)
+                if re_ids == ids:
+                    return text
+            logger.warning("Failed to find retokenizable sequence of length %d after 1000 attempts", length)
+            indices = torch.randint(0, allowed_ids.numel(), (length,), generator=rng)
+            return tokenizer.decode(allowed_ids[indices].tolist(), skip_special_tokens=False)
+
+        prompt = _sample_retokenizable(self.query_len)
+        target = _sample_retokenizable(self.target_len)
+
+        return SampleSpec(
+            messages=[{"role": "user", "content": prompt + OPTIM_PLACEHOLDER}],
+            target=target,
+        )
+
+
+class FixedSource(InstanceSource):
+    """Fixed prompt and target strings (same for all sample_ids)."""
+
+    type = "fixed"
+
+    def __init__(self, prompt: str = "", target: str = "", system_prompt: str | None = None):
+        self.prompt = prompt
+        self.target = target
+        self.system_prompt = system_prompt
+
+    def generate(self, sample_id: int, tokenizer: PreTrainedTokenizerBase) -> SampleSpec:
+        return SampleSpec(
+            messages=[{"role": "user", "content": self.prompt + OPTIM_PLACEHOLDER}],
+            target=self.target,
+            system_prompt=self.system_prompt,
+        )
+
+
+class ClearHarmSource(InstanceSource):
+    """Load harmful queries from the ClearHarm HuggingFace dataset."""
+
+    type = "clearharm"
+
+    def __init__(
+        self,
+        target: str = "Sure, here is",
+        dataset_name: str = "AlignmentResearch/ClearHarm",
+        config: str = "default",
+        split: str = "train",
+    ):
+        self.target = target
+        self.dataset_name = dataset_name
+        self.config = config
+        self.split = split
+        self._examples: list[str] | None = None
+
+    def _load(self) -> list[str]:
+        if self._examples is not None:
+            return self._examples
+        from datasets import load_dataset
+
+        ds = load_dataset(self.dataset_name, self.config, split=self.split)
+        self._examples = [row["content"][0] for row in ds]
+        logger.info("ClearHarmSource: loaded %d examples from %s", len(self._examples), self.dataset_name)
+        return self._examples
+
+    def generate(self, sample_id: int, tokenizer: PreTrainedTokenizerBase) -> SampleSpec:
+        examples = self._load()
+        idx = sample_id % len(examples)
+        query = examples[idx]
+        return SampleSpec(
+            messages=[{"role": "user", "content": f"{query} {OPTIM_PLACEHOLDER}"}],
+            target=self.target,
+        )
+
+
+# ---------------------------------------------------------------------------
+# Token layout
+# ---------------------------------------------------------------------------
+
+
+@dataclass
+class TokenLayoutResult:
+    """Output of applying a layout to a sequence."""
+
+    initial_ids: Tensor
+    optimizable_mask: Tensor
+
+
+class TokenLayout(_Registrable):
+    """Which positions in the optimizable region are free vs fixed."""
+
+    _registry: dict[str, type] = {}
+
+    @abstractmethod
+    def apply(self, optim_length: int, init_ids: Tensor, **ctx) -> TokenLayoutResult:
+        """Compute layout.  Extra optimizer state (model, tokenizer, embeds, …) via *ctx*."""
+
+
+class SuffixLayout(TokenLayout):
+    """All positions are optimizable (default)."""
+
+    type = "suffix"
+
+    def apply(self, optim_length: int, init_ids: Tensor, **ctx) -> TokenLayoutResult:
+        return TokenLayoutResult(
+            initial_ids=init_ids[:optim_length],
+            optimizable_mask=torch.ones(optim_length, dtype=torch.bool, device=init_ids.device),
+        )
+
+
+# ---------------------------------------------------------------------------
+# Init strategy
+# ---------------------------------------------------------------------------
+
+
+class InitStrategy(_Registrable):
+    """How to initialize optimizable token positions."""
+
+    _registry: dict[str, type] = {}
+
+    @abstractmethod
+    def initialize(
+        self,
+        optim_length: int,
+        tokenizer: PreTrainedTokenizerBase,
+        allowed_token_ids: Tensor,
+        *,
+        target_ids: Tensor | None = None,
+        device: torch.device | None = None,
+    ) -> Tensor:
+        """Return [optim_length] tensor of initial token IDs."""
+
+
+class RandomInit(InitStrategy):
+    """Sample random tokens from allowed vocabulary."""
+
+    type = "random"
+
+    def initialize(
+        self,
+        optim_length: int,
+        tokenizer: PreTrainedTokenizerBase,
+        allowed_token_ids: Tensor,
+        *,
+        target_ids: Tensor | None = None,
+        device: torch.device | None = None,
+    ) -> Tensor:
+        if device is None:
+            device = allowed_token_ids.device
+        indices = torch.randint(0, allowed_token_ids.numel(), (optim_length,), device=device)
+        return allowed_token_ids[indices]
+
+
+# ---------------------------------------------------------------------------
+# InputSpec — ties the three axes together
+# ---------------------------------------------------------------------------
+
+
+@dataclass
+class InputSpec:
+    """Compositional specification: source + layout + init."""
+
+    source: InstanceSource
+    layout: TokenLayout = field(default_factory=SuffixLayout)
+    init: InitStrategy = field(default_factory=RandomInit)
+
+    def to_dict(self) -> dict:
+        return {
+            "source": self.source.to_dict(),
+            "layout": self.layout.to_dict(),
+            "init": self.init.to_dict(),
+        }
+
+    @staticmethod
+    def from_dict(d: dict) -> InputSpec:
+        return InputSpec(
+            source=InstanceSource.from_dict(d["source"]),
+            layout=TokenLayout.from_dict(d["layout"]) if "layout" in d else SuffixLayout(),
+            init=InitStrategy.from_dict(d["init"]) if "init" in d else RandomInit(),
+        )
+
+    @staticmethod
+    def default() -> InputSpec:
+        return InputSpec(source=RandomSource(query_len=0, target_len=20))

claudini/methods/claude_demo/__init__.py

@@ -0,0 +1 @@
+

claudini/methods/claude_random/__init__.py

@@ -0,0 +1 @@
+

claudini/methods/claude_random/v1/__init__.py

@@ -0,0 +1,3 @@
+from .optimizer import ClaudeOptimizer
+
+__all__ = ["ClaudeOptimizer"]

claudini/methods/claude_random/v1/optimizer.py

@@ -0,0 +1,284 @@
+"""
+Claude optimizer: hybrid combining the best discrete GCG-family techniques.
+
+Combines:
+  1. Multi-restart (K=4) with batched gradient — from gcg_fast
+  2. ACG adaptive schedules: n_replace decay (5→1) + search_width ramp
+  3. LSGM gradient hooks (gamma=0.5) — from i_gcg, zero-cost on 7B+ models
+  4. Gradient momentum (mu=0.5) — from MAC, smooths noisy gradient signal
+  5. Best-ever buffer per restart — from ACG, stable gradient anchor
+  6. Patience + perturbation — from gcg_fast, escape local minima
+"""
+
+import logging
+
+import torch
+from torch import Tensor
+from transformers import PreTrainedModel, PreTrainedTokenizerBase
+
+from claudini.base import TokenOptimizer
+from claudini.tokens import sample_ids_from_grad
+
+logger = logging.getLogger("claudini")
+
+
+class ClaudeOptimizer(TokenOptimizer):
+    method_name = "claude_v1"
+
+    def __init__(
+        self,
+        model: PreTrainedModel,
+        tokenizer: PreTrainedTokenizerBase,
+        optim_length: int = 20,
+        # Multi-restart
+        num_starts: int = 4,
+        # ACG-style adaptive schedules
+        n_replace_max: int = 5,
+        n_replace_min: int = 1,
+        search_width_min: int = 64,
+        search_width_max: int = 256,
+        topk_per_position: int = 256,
+        # LSGM
+        lsgm_gamma: float = 0.5,
+        # Momentum
+        momentum: float = 0.5,
+        # Patience
+        patience: int = 50,
+        n_perturb: int = 3,
+        seed: int | None = None,
+        allow_non_ascii: bool = False,
+    ):
+        super().__init__(model, tokenizer, optim_length, seed, allow_non_ascii)
+        self.num_starts = num_starts
+        self.n_replace_max = n_replace_max
+        self.n_replace_min = n_replace_min
+        self.search_width_min = search_width_min
+        self.search_width_max = search_width_max
+        self.topk_per_position = topk_per_position
+        self.lsgm_gamma = lsgm_gamma
+        self.momentum = momentum
+        self.patience_limit = patience
+        self.n_perturb = n_perturb
+
+        # State (initialized in setup)
+        self.current_ids: Tensor | None = None
+        self.best_ids: Tensor | None = None
+        self.best_losses: list | None = None
+        self._restart_patience: list | None = None
+        self._momentum_buffer: list | None = None
+        self._lsgm_handles: list = []
+        self.max_flops: float | None = None
+
+    def _get_progress(self) -> float:
+        if self.max_flops is None or self.max_flops <= 0:
+            return 0.0
+        return min(1.0, self.flop_counter.total_flops / self.max_flops)
+
+    def _get_n_replace(self) -> int:
+        t = self._get_progress()
+        m = self.n_replace_max + t * (self.n_replace_min - self.n_replace_max)
+        return max(self.n_replace_min, int(round(m)))
+
+    def _get_search_width(self) -> int:
+        t = self._get_progress()
+        B = self.search_width_min + t * (self.search_width_max - self.search_width_min)
+        return max(1, int(round(B)))
+
+    # --- LSGM hooks ---
+
+    def _get_norm_modules(self):
+        norms = []
+        for name, module in self.model.named_modules():
+            if any(
+                p in name
+                for p in [
+                    "input_layernorm",
+                    "post_attention_layernorm",
+                    "pre_feedforward_layernorm",
+                    "post_feedforward_layernorm",
+                    ".ln_1",
+                    ".ln_2",
+                ]
+            ):
+                norms.append(module)
+        return norms
+
+    def _register_lsgm_hooks(self) -> list:
+        handles = []
+        gamma = self.lsgm_gamma
+        for module in self._get_norm_modules():
+
+            def hook(m, grad_input, grad_output, _gamma=gamma):
+                grad_input[0].data *= _gamma
+
+            handles.append(module.register_full_backward_hook(hook))
+        return handles
+
+    def _remove_hooks(self) -> None:
+        for h in self._lsgm_handles:
+            h.remove()
+        self._lsgm_handles.clear()
+
+    # --- Setup ---
+
+    def setup(self, prompt: str, target: str) -> None:
+        self._prepare_prompt(prompt, target)
+        K = self.num_starts
+
+        # Initialize K random restarts
+        ids_list = [self._init_optim_ids() for _ in range(K)]
+        self.current_ids = torch.stack(ids_list, dim=0)
+
+        # Evaluate initial losses
+        init_losses = self.compute_discrete_loss_batch(self.current_ids)
+        self.flop_counter.count_forward(self.total_seq_len, batch_size=K)
+        self.best_losses = init_losses.tolist()
+        self.best_ids = self.current_ids.clone()
+        self._restart_patience = [0] * K
+        self._momentum_buffer = [None] * K
+
+        # Register LSGM hooks
+        self._lsgm_handles = self._register_lsgm_hooks()
+        logger.info(
+            "Claude: K=%d restarts, LSGM(%d hooks, gamma=%.2f), momentum=%.2f",
+            K,
+            len(self._lsgm_handles),
+            self.lsgm_gamma,
+            self.momentum,
+        )
+
+    # --- Step ---
+
+    def step(self, step_num: int) -> tuple[float, float | None, str]:
+        K = self.num_starts
+        n_replace = self._get_n_replace()
+        search_width = self._get_search_width()
+
+        # 1. Batched gradient from best-ever suffixes (LSGM hooks fire automatically)
+        grads = self._compute_batched_gradient(self.best_ids)
+        self.flop_counter.count_forward_backward(self.total_seq_len, batch_size=K)
+
+        with torch.no_grad():
+            # 2. Apply momentum per restart
+            for k in range(K):
+                if self._momentum_buffer[k] is None:
+                    self._momentum_buffer[k] = grads[k].clone()
+                else:
+                    self._momentum_buffer[k] = self.momentum * self._momentum_buffer[k] + (1 - self.momentum) * grads[k]
+
+            # 3. Sample candidates per restart using momentum-smoothed gradient
+            all_candidates = []
+            restart_sizes = []
+            for k in range(K):
+                sampled = sample_ids_from_grad(
+                    self.best_ids[k],
+                    self._momentum_buffer[k],
+                    search_width,
+                    self.topk_per_position,
+                    n_replace,
+                    not_allowed_ids=self.not_allowed_ids,
+                )
+                all_candidates.append(sampled)
+                restart_sizes.append(sampled.shape[0])
+
+            all_candidates = torch.cat(all_candidates, dim=0)
+            total_candidates = sum(restart_sizes)
+
+            # 4. Batched evaluation
+            batch_losses = self.compute_discrete_loss_batch(all_candidates)
+            self.flop_counter.count_forward(self.total_seq_len, batch_size=total_candidates)
+
+            # 5. Per-restart: update best-ever
+            offset = 0
+            for k in range(K):
+                sz = restart_sizes[k]
+                restart_losses = batch_losses[offset : offset + sz]
+                best_idx = restart_losses.argmin().item()
+                candidate_loss = restart_losses[best_idx].item()
+
+                self.current_ids[k] = all_candidates[offset + best_idx]
+
+                if candidate_loss < self.best_losses[k]:
+                    self.best_losses[k] = candidate_loss
+                    self.best_ids[k] = self.current_ids[k].clone()
+                    self._restart_patience[k] = 0
+                else:
+                    self._restart_patience[k] += 1
+                offset += sz
+
+            # 6. Patience: perturb stalled restarts
+            for k in range(K):
+                if self._restart_patience[k] >= self.patience_limit:
+                    self._perturb_restart(k)
+                    self._restart_patience[k] = 0
+
+        # Log schedule values
+        self.log("n_replace", n_replace, prog_bar=True)
+        self.log("search_width", search_width)
+
+        # Return global best
+        best_k = min(range(K), key=lambda k: self.best_losses[k])
+        optim_str = self.tokenizer.decode(self.best_ids[best_k])
+        self._step_ids = self.best_ids[best_k]
+        return self.best_losses[best_k], None, optim_str
+
+    def _perturb_restart(self, k: int) -> None:
+        self.current_ids[k] = self.best_ids[k].clone()
+        positions = torch.randperm(self.optim_length, device=self.current_ids.device)[: self.n_perturb]
+        random_tokens = self.allowed_token_ids[
+            torch.randint(len(self.allowed_token_ids), (self.n_perturb,), device=self.current_ids.device)
+        ]
+        self.current_ids[k, positions] = random_tokens
+        self.best_ids[k] = self.current_ids[k].clone()
+        new_loss = self.compute_discrete_loss(self.current_ids[k])
+        self.flop_counter.count_forward(self.total_seq_len)
+        self.best_losses[k] = new_loss
+        # Reset momentum for this restart
+        self._momentum_buffer[k] = None
+
+    def _compute_batched_gradient(self, optim_ids: Tensor) -> Tensor:
+        K = optim_ids.shape[0]
+        embedding_layer = self.embedding_layer
+
+        optim_ids_onehot = torch.nn.functional.one_hot(
+            optim_ids,
+            num_classes=embedding_layer.num_embeddings,
+        ).to(self.model.device, self.model.dtype)
+        optim_ids_onehot.requires_grad_(True)
+
+        optim_embeds = optim_ids_onehot @ embedding_layer.weight
+
+        input_embeds = torch.cat(
+            [
+                self.before_embeds.expand(K, -1, -1),
+                optim_embeds,
+                self.after_embeds.expand(K, -1, -1),
+                self.target_embeds.expand(K, -1, -1),
+            ],
+            dim=1,
+        )
+
+        output = self.model(inputs_embeds=input_embeds)
+        logits = output.logits
+
+        shift = input_embeds.shape[1] - self.target_ids.shape[1]
+        target_len = self.target_ids.shape[1]
+        shift_logits = logits[..., shift - 1 : shift - 1 + target_len, :].contiguous()
+
+        target_expanded = self.target_ids.expand(K, -1)
+        losses = torch.nn.functional.cross_entropy(
+            shift_logits.view(-1, shift_logits.size(-1)),
+            target_expanded.reshape(-1),
+            reduction="none",
+        )
+        total_loss = losses.sum()
+
+        grad = torch.autograd.grad(outputs=[total_loss], inputs=[optim_ids_onehot])[0]
+        return grad
+
+    def run(self, prompt, target, num_steps, max_flops=None, max_time=None, **kwargs):
+        self.max_flops = max_flops
+        try:
+            return super().run(prompt, target, num_steps, max_flops=max_flops, max_time=max_time, **kwargs)
+        finally:
+            self._remove_hooks()

claudini/methods/claude_random/v10/__init__.py

@@ -0,0 +1,3 @@
+from .optimizer import ClaudeV10Optimizer
+
+__all__ = ["ClaudeV10Optimizer"]

claudini/methods/claude_random/v10/optimizer.py

@@ -0,0 +1,39 @@
+"""
+Claude v10 optimizer: ADC + LSGM gamma=0.3.
+
+Base: v6 (ADC + LSGM gamma=0.5) — avg 0.80 on Qwen.
+Change: More aggressive gradient scaling (gamma=0.3 vs 0.5).
+
+Motivation: gamma=0.5 was borrowed from i_gcg's default. On continuous
+optimization (ADC), stronger gradient scaling might help even more —
+pushing the skip-connection signal to dominate even further.
+"""
+
+import logging
+
+from transformers import PreTrainedModel, PreTrainedTokenizerBase
+
+from claudini.methods.claude_random.v6 import ClaudeV6Optimizer
+
+logger = logging.getLogger("claudini")
+
+
+class ClaudeV10Optimizer(ClaudeV6Optimizer):
+    method_name = "claude_v10"
+
+    def __init__(
+        self,
+        model: PreTrainedModel,
+        tokenizer: PreTrainedTokenizerBase,
+        optim_length: int = 20,
+        lr: float = 10.0,
+        momentum: float = 0.99,
+        ema_alpha: float = 0.01,
+        num_starts: int = 16,
+        lsgm_gamma: float = 0.3,
+        seed: int | None = None,
+        allow_non_ascii: bool = False,
+    ):
+        super().__init__(
+            model, tokenizer, optim_length, lr, momentum, ema_alpha, num_starts, lsgm_gamma, seed, allow_non_ascii
+        )

claudini/methods/claude_random/v100/__init__.py

@@ -0,0 +1,3 @@
+from claudini.methods.claude_random.v100.optimizer import ClaudeV100Optimizer
+
+__all__ = ["ClaudeV100Optimizer"]

claudini/methods/claude_random/v100/optimizer.py

@@ -0,0 +1,32 @@
+"""Claude v100: Nesterov + patience=50 + restore-from-best. Full combination of best techniques."""
+
+import torch
+from transformers import PreTrainedModel, PreTrainedTokenizerBase
+
+from claudini.methods.claude_random.v90 import ClaudeV90Optimizer
+
+
+class ClaudeV100Optimizer(ClaudeV90Optimizer):
+    method_name = "claude_v100"
+
+    def __init__(
+        self,
+        model: PreTrainedModel,
+        tokenizer: PreTrainedTokenizerBase,
+        optim_length: int = 20,
+        lr: float = 10.0,
+        momentum: float = 0.99,
+        ema_alpha: float = 0.01,
+        num_starts: int = 8,
+        lsgm_gamma: float = 0.70,
+        seed: int | None = None,
+        allow_non_ascii: bool = False,
+    ):
+        super().__init__(
+            model, tokenizer, optim_length, lr, momentum, ema_alpha, num_starts, lsgm_gamma, seed, allow_non_ascii
+        )
+        self.patience = 50
+
+    def setup(self, prompt: str, target: str) -> None:
+        super().setup(prompt, target)
+        self.optimizer = torch.optim.SGD([self.soft_opt], lr=self.lr, momentum=self.momentum, nesterov=True)

claudini/methods/claude_random/v101/__init__.py

@@ -0,0 +1,3 @@
+from claudini.methods.claude_random.v101.optimizer import ClaudeV101Optimizer
+
+__all__ = ["ClaudeV101Optimizer"]

claudini/methods/claude_random/v101/optimizer.py

@@ -0,0 +1,28 @@
+"""Claude v101: Patience=50 + n_perturb=3 K=8 γ=0.70. Fewer perturbed positions for less disruption."""
+
+from transformers import PreTrainedModel, PreTrainedTokenizerBase
+
+from claudini.methods.claude_random.v86 import ClaudeV86Optimizer
+
+
+class ClaudeV101Optimizer(ClaudeV86Optimizer):
+    method_name = "claude_v101"
+
+    def __init__(
+        self,
+        model: PreTrainedModel,
+        tokenizer: PreTrainedTokenizerBase,
+        optim_length: int = 20,
+        lr: float = 10.0,
+        momentum: float = 0.99,
+        ema_alpha: float = 0.01,
+        num_starts: int = 8,
+        lsgm_gamma: float = 0.70,
+        seed: int | None = None,
+        allow_non_ascii: bool = False,
+    ):
+        super().__init__(
+            model, tokenizer, optim_length, lr, momentum, ema_alpha, num_starts, lsgm_gamma, seed, allow_non_ascii
+        )
+        self.patience = 50
+        self.n_perturb = 3

claudini/methods/claude_random/v102/__init__.py

@@ -0,0 +1,3 @@
+from claudini.methods.claude_random.v102.optimizer import ClaudeV102Optimizer
+
+__all__ = ["ClaudeV102Optimizer"]

claudini/methods/claude_random/v102/optimizer.py

@@ -0,0 +1,28 @@
+"""Claude v102: Patience=50 + n_perturb=2 K=8 γ=0.70. Minimal perturbation for gentler escape."""
+
+from transformers import PreTrainedModel, PreTrainedTokenizerBase
+
+from claudini.methods.claude_random.v86 import ClaudeV86Optimizer
+
+
+class ClaudeV102Optimizer(ClaudeV86Optimizer):
+    method_name = "claude_v102"
+
+    def __init__(
+        self,
+        model: PreTrainedModel,
+        tokenizer: PreTrainedTokenizerBase,
+        optim_length: int = 20,
+        lr: float = 10.0,
+        momentum: float = 0.99,
+        ema_alpha: float = 0.01,
+        num_starts: int = 8,
+        lsgm_gamma: float = 0.70,
+        seed: int | None = None,
+        allow_non_ascii: bool = False,
+    ):
+        super().__init__(
+            model, tokenizer, optim_length, lr, momentum, ema_alpha, num_starts, lsgm_gamma, seed, allow_non_ascii
+        )
+        self.patience = 50
+        self.n_perturb = 2

claudini/methods/claude_random/v103/__init__.py

@@ -0,0 +1,3 @@
+from claudini.methods.claude_random.v103.optimizer import ClaudeV103Optimizer
+
+__all__ = ["ClaudeV103Optimizer"]

claudini/methods/claude_random/v103/optimizer.py

@@ -0,0 +1,33 @@
+"""Claude v103: Nesterov + patience=50 + n_perturb=3 K=8 γ=0.70. Nesterov with fewer perturbed positions."""
+
+import torch
+from transformers import PreTrainedModel, PreTrainedTokenizerBase
+
+from claudini.methods.claude_random.v86 import ClaudeV86Optimizer
+
+
+class ClaudeV103Optimizer(ClaudeV86Optimizer):
+    method_name = "claude_v103"
+
+    def __init__(
+        self,
+        model: PreTrainedModel,
+        tokenizer: PreTrainedTokenizerBase,
+        optim_length: int = 20,
+        lr: float = 10.0,
+        momentum: float = 0.99,
+        ema_alpha: float = 0.01,
+        num_starts: int = 8,
+        lsgm_gamma: float = 0.70,
+        seed: int | None = None,
+        allow_non_ascii: bool = False,
+    ):
+        super().__init__(
+            model, tokenizer, optim_length, lr, momentum, ema_alpha, num_starts, lsgm_gamma, seed, allow_non_ascii
+        )
+        self.patience = 50
+        self.n_perturb = 3
+
+    def setup(self, prompt: str, target: str) -> None:
+        super().setup(prompt, target)
+        self.optimizer = torch.optim.SGD([self.soft_opt], lr=self.lr, momentum=self.momentum, nesterov=True)

claudini/methods/claude_random/v104/__init__.py

@@ -0,0 +1,3 @@
+from claudini.methods.claude_random.v104.optimizer import ClaudeV104Optimizer
+
+__all__ = ["ClaudeV104Optimizer"]

claudini/methods/claude_random/v104/optimizer.py

@@ -0,0 +1,27 @@
+"""Claude v104: Patience=50 + restore-from-best K=8 γ=0.70. Aggressive perturbation with restore-from-best."""
+
+from transformers import PreTrainedModel, PreTrainedTokenizerBase
+
+from claudini.methods.claude_random.v90 import ClaudeV90Optimizer
+
+
+class ClaudeV104Optimizer(ClaudeV90Optimizer):
+    method_name = "claude_v104"
+
+    def __init__(
+        self,
+        model: PreTrainedModel,
+        tokenizer: PreTrainedTokenizerBase,
+        optim_length: int = 20,
+        lr: float = 10.0,
+        momentum: float = 0.99,
+        ema_alpha: float = 0.01,
+        num_starts: int = 8,
+        lsgm_gamma: float = 0.70,
+        seed: int | None = None,
+        allow_non_ascii: bool = False,
+    ):
+        super().__init__(
+            model, tokenizer, optim_length, lr, momentum, ema_alpha, num_starts, lsgm_gamma, seed, allow_non_ascii
+        )
+        self.patience = 50

claudini/methods/claude_random/v105/__init__.py

@@ -0,0 +1,3 @@
+from claudini.methods.claude_random.v105.optimizer import ClaudeV105Optimizer
+
+__all__ = ["ClaudeV105Optimizer"]

claudini/methods/claude_random/v105/optimizer.py

@@ -0,0 +1,103 @@
+"""Claude v105: Adaptive patience per restart. Restarts that are improving get more patience; stagnant ones get less."""
+
+import logging
+
+import torch
+from transformers import PreTrainedModel, PreTrainedTokenizerBase
+
+from claudini.methods.claude_random.v86 import ClaudeV86Optimizer
+
+logger = logging.getLogger("claudini")
+
+
+class ClaudeV105Optimizer(ClaudeV86Optimizer):
+    method_name = "claude_v105"
+
+    def __init__(
+        self,
+        model: PreTrainedModel,
+        tokenizer: PreTrainedTokenizerBase,
+        optim_length: int = 20,
+        lr: float = 10.0,
+        momentum: float = 0.99,
+        ema_alpha: float = 0.01,
+        num_starts: int = 8,
+        lsgm_gamma: float = 0.70,
+        seed: int | None = None,
+        allow_non_ascii: bool = False,
+    ):
+        super().__init__(
+            model, tokenizer, optim_length, lr, momentum, ema_alpha, num_starts, lsgm_gamma, seed, allow_non_ascii
+        )
+        self.patience = 50  # base patience
+        self._loss_ema = None
+        self._adaptive_patience = None
+
+    def setup(self, prompt: str, target: str) -> None:
+        super().setup(prompt, target)
+        K = self.num_starts
+        device = self.soft_opt.device
+        self._loss_ema = torch.full((K,), float("inf"), device=device)
+        self._prev_loss_ema = torch.full((K,), float("inf"), device=device)
+        self._adaptive_patience = torch.full((K,), self.patience, dtype=torch.float32, device=device)
+
+    def step(self, step_num: int) -> tuple[float, float | None, str]:
+        # Run v26 step (skip v86's step to implement our own stagnation logic)
+        result = super(ClaudeV86Optimizer, self).step(step_num)
+
+        with torch.no_grad():
+            # Get per-restart discrete losses
+            all_ids = self.soft_opt.data.argmax(dim=-1)
+            losses = self.compute_discrete_loss_batch(all_ids)
+            self.flop_counter.count_forward(self.total_seq_len, batch_size=self.num_starts)
+
+            # Update loss EMA
+            if self._loss_ema[0] == float("inf"):
+                self._loss_ema = losses.clone()
+                self._prev_loss_ema = losses.clone()
+            else:
+                self._prev_loss_ema = self._loss_ema.clone()
+                self._loss_ema = 0.9 * self._loss_ema + 0.1 * losses
+
+            # Adapt patience per restart based on EMA trend
+            improving = self._loss_ema < self._prev_loss_ema  # EMA is decreasing
+            for k in range(self.num_starts):
+                if improving[k]:
+                    self._adaptive_patience[k] = min(self._adaptive_patience[k] * 1.5, self.patience * 3.0)
+                else:
+                    self._adaptive_patience[k] = max(self._adaptive_patience[k] * 0.5, 20.0)
+
+            # Track stagnation
+            improved = losses < self._best_per_restart
+            self._best_per_restart = torch.where(improved, losses, self._best_per_restart)
+            self._stagnant_count = torch.where(
+                improved, torch.zeros_like(self._stagnant_count), self._stagnant_count + 1
+            )
+
+            # Perturb stagnant restarts using adaptive patience
+            stagnant_mask = self._stagnant_count >= self._adaptive_patience.long()
+            if stagnant_mask.any():
+                n_stagnant = stagnant_mask.sum().item()
+                L, V = self.soft_opt.data.shape[1], self.soft_opt.data.shape[2]
+
+                for k in range(self.num_starts):
+                    if stagnant_mask[k]:
+                        positions = torch.randperm(L, device=self.soft_opt.device)[: self.n_perturb]
+                        self.soft_opt.data[k, positions] = 0.0
+                        rand_tokens = torch.randint(0, V, (self.n_perturb,), device=self.soft_opt.device)
+                        self.soft_opt.data[k, positions, rand_tokens] = 10.0
+
+                # Reset stagnation counter and momentum for perturbed restarts
+                self._stagnant_count[stagnant_mask] = 0
+                self._adaptive_patience[stagnant_mask] = self.patience  # reset to base
+                if self.optimizer.state:
+                    for group in self.optimizer.param_groups:
+                        for p in group["params"]:
+                            if p in self.optimizer.state:
+                                buf = self.optimizer.state[p].get("momentum_buffer")
+                                if buf is not None:
+                                    buf[stagnant_mask] = 0.0
+
+                self.log("perturbed", n_stagnant)
+
+        return result

claudini/methods/claude_random/v106/__init__.py

@@ -0,0 +1,3 @@
+from claudini.methods.claude_random.v106.optimizer import ClaudeV106Optimizer
+
+__all__ = ["ClaudeV106Optimizer"]

claudini/methods/claude_random/v106/optimizer.py

@@ -0,0 +1,94 @@
+"""Claude v106: Global best injection. When a restart stagnates, inject global best + perturb if it's much worse."""
+
+import logging
+
+import torch
+from transformers import PreTrainedModel, PreTrainedTokenizerBase
+
+from claudini.methods.claude_random.v86 import ClaudeV86Optimizer
+
+logger = logging.getLogger("claudini")
+
+
+class ClaudeV106Optimizer(ClaudeV86Optimizer):
+    method_name = "claude_v106"
+
+    def __init__(
+        self,
+        model: PreTrainedModel,
+        tokenizer: PreTrainedTokenizerBase,
+        optim_length: int = 20,
+        lr: float = 10.0,
+        momentum: float = 0.99,
+        ema_alpha: float = 0.01,
+        num_starts: int = 8,
+        lsgm_gamma: float = 0.70,
+        seed: int | None = None,
+        allow_non_ascii: bool = False,
+    ):
+        super().__init__(
+            model, tokenizer, optim_length, lr, momentum, ema_alpha, num_starts, lsgm_gamma, seed, allow_non_ascii
+        )
+        self.patience = 50
+        self._global_best_soft = None
+        self._global_best_discrete_loss = float("inf")
+
+    def setup(self, prompt: str, target: str) -> None:
+        super().setup(prompt, target)
+        self._global_best_soft = self.soft_opt.data[0].clone()
+        self._global_best_discrete_loss = float("inf")
+
+    def step(self, step_num: int) -> tuple[float, float | None, str]:
+        # Run v26 step (skip v86's step to implement our own stagnation logic)
+        result = super(ClaudeV86Optimizer, self).step(step_num)
+
+        with torch.no_grad():
+            # Get per-restart discrete losses
+            all_ids = self.soft_opt.data.argmax(dim=-1)
+            losses = self.compute_discrete_loss_batch(all_ids)
+            self.flop_counter.count_forward(self.total_seq_len, batch_size=self.num_starts)
+
+            # Track global best across all restarts
+            best_k = losses.argmin().item()
+            if losses[best_k].item() < self._global_best_discrete_loss:
+                self._global_best_discrete_loss = losses[best_k].item()
+                self._global_best_soft = self.soft_opt.data[best_k].clone()
+
+            # Track stagnation
+            improved = losses < self._best_per_restart
+            self._best_per_restart = torch.where(improved, losses, self._best_per_restart)
+            self._stagnant_count = torch.where(
+                improved, torch.zeros_like(self._stagnant_count), self._stagnant_count + 1
+            )
+
+            # Perturb stagnant restarts
+            stagnant_mask = self._stagnant_count >= self.patience
+            if stagnant_mask.any():
+                n_stagnant = stagnant_mask.sum().item()
+                L, V = self.soft_opt.data.shape[1], self.soft_opt.data.shape[2]
+
+                for k in range(self.num_starts):
+                    if stagnant_mask[k]:
+                        # If this restart's best is >2x the global best, inject global best
+                        if self._best_per_restart[k].item() > 2.0 * self._global_best_discrete_loss:
+                            self.soft_opt.data[k] = self._global_best_soft.clone()
+
+                        # Perturb n_perturb positions (on top of injected or current state)
+                        positions = torch.randperm(L, device=self.soft_opt.device)[: self.n_perturb]
+                        self.soft_opt.data[k, positions] = 0.0
+                        rand_tokens = torch.randint(0, V, (self.n_perturb,), device=self.soft_opt.device)
+                        self.soft_opt.data[k, positions, rand_tokens] = 10.0
+
+                # Reset stagnation counter and momentum for perturbed restarts
+                self._stagnant_count[stagnant_mask] = 0
+                if self.optimizer.state:
+                    for group in self.optimizer.param_groups:
+                        for p in group["params"]:
+                            if p in self.optimizer.state:
+                                buf = self.optimizer.state[p].get("momentum_buffer")
+                                if buf is not None:
+                                    buf[stagnant_mask] = 0.0
+
+                self.log("perturbed", n_stagnant)
+
+        return result

claudini/methods/claude_random/v107/__init__.py

@@ -0,0 +1,3 @@
+from claudini.methods.claude_random.v107.optimizer import ClaudeV107Optimizer
+
+__all__ = ["ClaudeV107Optimizer"]

claudini/methods/claude_random/v107/optimizer.py

@@ -0,0 +1,31 @@
+"""Claude v107: patience=50 + lr=12. Test if higher lr helps with shorter patience."""
+
+import logging
+
+from transformers import PreTrainedModel, PreTrainedTokenizerBase
+
+from claudini.methods.claude_random.v86 import ClaudeV86Optimizer
+
+logger = logging.getLogger("claudini")
+
+
+class ClaudeV107Optimizer(ClaudeV86Optimizer):
+    method_name = "claude_v107"
+
+    def __init__(
+        self,
+        model: PreTrainedModel,
+        tokenizer: PreTrainedTokenizerBase,
+        optim_length: int = 20,
+        lr: float = 12.0,
+        momentum: float = 0.99,
+        ema_alpha: float = 0.01,
+        num_starts: int = 8,
+        lsgm_gamma: float = 0.70,
+        seed: int | None = None,
+        allow_non_ascii: bool = False,
+    ):
+        super().__init__(
+            model, tokenizer, optim_length, lr, momentum, ema_alpha, num_starts, lsgm_gamma, seed, allow_non_ascii
+        )
+        self.patience = 50

claudini/methods/claude_random/v108/__init__.py

@@ -0,0 +1,3 @@
+from claudini.methods.claude_random.v108.optimizer import ClaudeV108Optimizer
+
+__all__ = ["ClaudeV108Optimizer"]

claudini/methods/claude_random/v108/optimizer.py

@@ -0,0 +1,105 @@
+"""Claude v108: Adaptive perturbation strength. Escalating n_perturb for repeatedly stagnant restarts."""
+
+import logging
+
+import torch
+from transformers import PreTrainedModel, PreTrainedTokenizerBase
+
+from claudini.methods.claude_random.v86 import ClaudeV86Optimizer
+
+logger = logging.getLogger("claudini")
+
+
+class ClaudeV108Optimizer(ClaudeV86Optimizer):
+    method_name = "claude_v108"
+
+    def __init__(
+        self,
+        model: PreTrainedModel,
+        tokenizer: PreTrainedTokenizerBase,
+        optim_length: int = 20,
+        lr: float = 10.0,
+        momentum: float = 0.99,
+        ema_alpha: float = 0.01,
+        num_starts: int = 8,
+        lsgm_gamma: float = 0.70,
+        seed: int | None = None,
+        allow_non_ascii: bool = False,
+    ):
+        super().__init__(
+            model, tokenizer, optim_length, lr, momentum, ema_alpha, num_starts, lsgm_gamma, seed, allow_non_ascii
+        )
+        self.patience = 50
+        self._perturb_count = None
+        self._prev_best = None
+
+    def setup(self, prompt: str, target: str) -> None:
+        super().setup(prompt, target)
+        K = self.num_starts
+        device = self.soft_opt.device
+        self._perturb_count = torch.zeros(K, dtype=torch.long, device=device)
+        self._prev_best = torch.full((K,), float("inf"), device=device)
+
+    def step(self, step_num: int) -> tuple[float, float | None, str]:
+        # Run v26 step (skip v86's step to implement our own stagnation logic)
+        result = super(ClaudeV86Optimizer, self).step(step_num)
+
+        with torch.no_grad():
+            # Get per-restart discrete losses
+            all_ids = self.soft_opt.data.argmax(dim=-1)
+            losses = self.compute_discrete_loss_batch(all_ids)
+            self.flop_counter.count_forward(self.total_seq_len, batch_size=self.num_starts)
+
+            # Track stagnation
+            improved = losses < self._best_per_restart
+            self._best_per_restart = torch.where(improved, losses, self._best_per_restart)
+            self._stagnant_count = torch.where(
+                improved, torch.zeros_like(self._stagnant_count), self._stagnant_count + 1
+            )
+
+            # Reset perturb count if restart improves significantly (>20% drop)
+            for k in range(self.num_starts):
+                if self._prev_best[k] != float("inf") and self._best_per_restart[k] < 0.8 * self._prev_best[k]:
+                    self._perturb_count[k] = 0
+                    self._prev_best[k] = self._best_per_restart[k].clone()
+                elif improved[k]:
+                    self._prev_best[k] = self._best_per_restart[k].clone()
+
+            # Perturb stagnant restarts with adaptive strength
+            stagnant_mask = self._stagnant_count >= self.patience
+            if stagnant_mask.any():
+                n_stagnant = stagnant_mask.sum().item()
+                L, V = self.soft_opt.data.shape[1], self.soft_opt.data.shape[2]
+
+                for k in range(self.num_starts):
+                    if stagnant_mask[k]:
+                        # Adaptive perturbation strength
+                        pc = self._perturb_count[k].item()
+                        if pc == 0:
+                            n_perturb = 3  # gentle
+                        elif pc == 1:
+                            n_perturb = 4  # normal
+                        else:
+                            n_perturb = 6  # aggressive
+
+                        n_perturb = min(n_perturb, L)  # don't exceed sequence length
+                        positions = torch.randperm(L, device=self.soft_opt.device)[:n_perturb]
+                        self.soft_opt.data[k, positions] = 0.0
+                        rand_tokens = torch.randint(0, V, (n_perturb,), device=self.soft_opt.device)
+                        self.soft_opt.data[k, positions, rand_tokens] = 10.0
+
+                        self._perturb_count[k] += 1
+
+                # Reset stagnation counter and momentum for perturbed restarts
+                self._stagnant_count[stagnant_mask] = 0
+                if self.optimizer.state:
+                    for group in self.optimizer.param_groups:
+                        for p in group["params"]:
+                            if p in self.optimizer.state:
+                                buf = self.optimizer.state[p].get("momentum_buffer")
+                                if buf is not None:
+                                    buf[stagnant_mask] = 0.0
+
+                self.log("perturbed", n_stagnant)
+
+        return result

claudini/methods/claude_random/v109/__init__.py

@@ -0,0 +1,3 @@
+from claudini.methods.claude_random.v109.optimizer import ClaudeV109Optimizer
+
+__all__ = ["ClaudeV109Optimizer"]

claudini/methods/claude_random/v109/optimizer.py

@@ -0,0 +1,43 @@
+"""Claude v109: Momentum schedule (warmup). Momentum starts at 0.95, linearly increases to 0.99 over the run."""
+
+import logging
+
+from transformers import PreTrainedModel, PreTrainedTokenizerBase
+
+from claudini.methods.claude_random.v86 import ClaudeV86Optimizer
+
+logger = logging.getLogger("claudini")
+
+
+class ClaudeV109Optimizer(ClaudeV86Optimizer):
+    method_name = "claude_v109"
+
+    def __init__(
+        self,
+        model: PreTrainedModel,
+        tokenizer: PreTrainedTokenizerBase,
+        optim_length: int = 20,
+        lr: float = 10.0,
+        momentum: float = 0.99,
+        ema_alpha: float = 0.01,
+        num_starts: int = 8,
+        lsgm_gamma: float = 0.70,
+        seed: int | None = None,
+        allow_non_ascii: bool = False,
+    ):
+        super().__init__(
+            model, tokenizer, optim_length, lr, momentum, ema_alpha, num_starts, lsgm_gamma, seed, allow_non_ascii
+        )
+        self.patience = 50
+        self._mom_start = 0.95
+        self._mom_end = 0.99
+        self._mom_warmup_steps = 5000  # approximate total steps for full warmup
+
+    def step(self, step_num: int) -> tuple[float, float | None, str]:
+        # Update momentum schedule before parent step
+        progress = min(1.0, step_num / self._mom_warmup_steps)
+        current_mom = self._mom_start + (self._mom_end - self._mom_start) * progress
+        for group in self.optimizer.param_groups:
+            group["momentum"] = current_mom
+
+        return super().step(step_num)

claudini/methods/claude_random/v11/__init__.py

@@ -0,0 +1,3 @@
+from .optimizer import ClaudeV11Optimizer
+
+__all__ = ["ClaudeV11Optimizer"]

claudini/methods/claude_random/v11/optimizer.py

@@ -0,0 +1,174 @@
+"""
+Claude v11 optimizer: ADC + LSGM + LILA.
+
+Base: v6 (ADC + LSGM gamma=0.5) — avg 0.80 on Qwen.
+Addition: LILA gradient direction replacement at intermediate layer.
+
+LILA replaces the backward gradient at an intermediate transformer layer's
+target position with a direction pointing from current activations toward
+initial activations. This guides optimization toward the initial activation
+space. Combined with LSGM's gradient scaling, we get two complementary
+gradient modifications on ADC's continuous optimization.
+
+Note: LILA requires an extra forward pass per step to capture current
+activations, costing ~50% more FLOPs per step (so fewer total steps).
+"""
+
+import logging
+
+import torch
+from torch import Tensor
+from transformers import PreTrainedModel, PreTrainedTokenizerBase
+
+from claudini.methods.original.adc import ADCOptimizer
+
+logger = logging.getLogger("claudini")
+
+
+class ClaudeV11Optimizer(ADCOptimizer):
+    method_name = "claude_v11"
+
+    def __init__(
+        self,
+        model: PreTrainedModel,
+        tokenizer: PreTrainedTokenizerBase,
+        optim_length: int = 20,
+        lr: float = 10.0,
+        momentum: float = 0.99,
+        ema_alpha: float = 0.01,
+        num_starts: int = 16,
+        lsgm_gamma: float = 0.5,
+        lila_layer: int | None = None,
+        seed: int | None = None,
+        allow_non_ascii: bool = False,
+    ):
+        super().__init__(model, tokenizer, optim_length, lr, momentum, ema_alpha, num_starts, seed, allow_non_ascii)
+        self.lsgm_gamma = lsgm_gamma
+        self._lila_layer_idx = lila_layer
+        self._lsgm_handles: list = []
+        self._lila_module = None
+        self.act_init: Tensor | None = None
+
+    def _get_transformer_blocks(self):
+        if hasattr(self.model, "model") and hasattr(self.model.model, "layers"):
+            return self.model.model.layers
+        if hasattr(self.model, "transformer") and hasattr(self.model.transformer, "h"):
+            return self.model.transformer.h
+        raise ValueError(f"Cannot find transformer blocks for {type(self.model)}")
+
+    def _get_norm_modules(self):
+        norms = []
+        for name, module in self.model.named_modules():
+            if any(
+                p in name
+                for p in [
+                    "input_layernorm",
+                    "post_attention_layernorm",
+                    "pre_feedforward_layernorm",
+                    "post_feedforward_layernorm",
+                    ".ln_1",
+                    ".ln_2",
+                ]
+            ):
+                norms.append(module)
+        return norms
+
+    def _register_lsgm_hooks(self) -> list:
+        handles = []
+        gamma = self.lsgm_gamma
+        for module in self._get_norm_modules():
+
+            def hook(m, grad_input, grad_output, _gamma=gamma):
+                grad_input[0].data *= _gamma
+
+            handles.append(module.register_full_backward_hook(hook))
+        return handles
+
+    def _remove_hooks(self) -> None:
+        for h in self._lsgm_handles:
+            h.remove()
+        self._lsgm_handles.clear()
+
+    def _capture_activations(self, optim_ids: Tensor) -> Tensor:
+        """Forward pass to capture activations at LILA layer."""
+        act = {}
+
+        def fwd_hook(m, inp, out):
+            act["val"] = inp[0].detach().clone()
+
+        handle = self._lila_module.register_forward_hook(fwd_hook)
+        with torch.no_grad():
+            # Build input for a single representative (first restart's argmax)
+            ids = optim_ids[:1] if optim_ids.dim() == 2 else optim_ids.unsqueeze(0)
+            optim_embeds = self.embedding_layer(ids).to(self.model_dtype)
+            input_embeds = torch.cat(
+                [self.before_embeds, optim_embeds, self.after_embeds, self.target_embeds],
+                dim=1,
+            )
+            self.model(inputs_embeds=input_embeds)
+        handle.remove()
+        self.flop_counter.count_forward(self.total_seq_len)
+        return act["val"]
+
+    def _get_target_token_position(self) -> int:
+        return self.n_before_tokens + self.optim_length + self.n_after_tokens
+
+    def setup(self, prompt: str, target: str) -> None:
+        super().setup(prompt, target)
+
+        # LSGM hooks
+        self._lsgm_handles = self._register_lsgm_hooks()
+
+        # LILA setup
+        blocks = self._get_transformer_blocks()
+        layer_idx = self._lila_layer_idx if self._lila_layer_idx is not None else len(blocks) // 2
+        self._lila_module = blocks[layer_idx]
+
+        # Capture initial activations using random init ids
+        init_ids = self.soft_opt.data[:1].argmax(dim=-1)  # [1, L]
+        self.act_init = self._capture_activations(init_ids)
+
+        logger.info(
+            "Claude v11: ADC + LSGM(%d hooks, gamma=%.2f) + LILA(layer=%d), K=%d",
+            len(self._lsgm_handles),
+            self.lsgm_gamma,
+            layer_idx,
+            self.num_starts,
+        )
+
+    def step(self, step_num: int) -> tuple[float, float | None, str]:
+        # Capture current activations for LILA
+        curr_ids = self.soft_opt.data[:1].argmax(dim=-1)
+        act_curr = self._capture_activations(curr_ids)
+
+        # Register LILA hook (skip step 0)
+        lila_handle = None
+        if step_num > 0:
+            tok_pos = self._get_target_token_position()
+            diff = self.act_init - act_curr
+            model_dtype = self.model_dtype
+
+            def lila_hook(m, grad_input, grad_output):
+                grad_at_tok = grad_input[0][:, tok_pos : tok_pos + 1, :]
+                magnitude = grad_at_tok.norm(p=2, dim=(1, 2), keepdim=True)
+                diff_at_tok = diff[:, tok_pos : tok_pos + 1, :].float()
+                diff_norm = diff_at_tok.norm(p=2, dim=(1, 2), keepdim=True).clamp(min=1e-12)
+                direction = diff_at_tok / diff_norm
+                grad_input[0].data[:, tok_pos : tok_pos + 1, :] = (magnitude * direction).to(model_dtype)
+
+            lila_handle = self._lila_module.register_full_backward_hook(lila_hook)
+
+        # Standard ADC step (LSGM hooks + LILA hook fire during backward)
+        result = super().step(step_num)
+
+        # Remove LILA hook
+        if lila_handle is not None:
+            lila_handle.remove()
+
+        return result
+
+    def run(self, prompt, target, num_steps, max_flops=None, max_time=None, **kwargs):
+        try:
+            return super().run(prompt, target, num_steps, max_flops=max_flops, max_time=max_time, **kwargs)
+        finally:
+            self._remove_hooks()

claudini/methods/claude_random/v110/__init__.py

@@ -0,0 +1,3 @@
+from claudini.methods.claude_random.v110.optimizer import ClaudeV110Optimizer
+
+__all__ = ["ClaudeV110Optimizer"]

claudini/methods/claude_random/v110/optimizer.py

@@ -0,0 +1,81 @@
+"""Claude v110: Ensemble voting. Every 500 steps, set the worst restart to consensus argmax tokens."""
+
+import logging
+
+import torch
+from transformers import PreTrainedModel, PreTrainedTokenizerBase
+
+from claudini.methods.claude_random.v86 import ClaudeV86Optimizer
+
+logger = logging.getLogger("claudini")
+
+
+class ClaudeV110Optimizer(ClaudeV86Optimizer):
+    method_name = "claude_v110"
+
+    def __init__(
+        self,
+        model: PreTrainedModel,
+        tokenizer: PreTrainedTokenizerBase,
+        optim_length: int = 20,
+        lr: float = 10.0,
+        momentum: float = 0.99,
+        ema_alpha: float = 0.01,
+        num_starts: int = 8,
+        lsgm_gamma: float = 0.70,
+        seed: int | None = None,
+        allow_non_ascii: bool = False,
+    ):
+        super().__init__(
+            model, tokenizer, optim_length, lr, momentum, ema_alpha, num_starts, lsgm_gamma, seed, allow_non_ascii
+        )
+        self.patience = 50
+        self.vote_interval = 500
+
+    def step(self, step_num: int) -> tuple[float, float | None, str]:
+        result = super().step(step_num)
+
+        # Ensemble voting every vote_interval steps
+        if step_num > 0 and step_num % self.vote_interval == 0:
+            with torch.no_grad():
+                K = self.num_starts
+                L, V = self.soft_opt.data.shape[1], self.soft_opt.data.shape[2]
+
+                # Get argmax tokens from all restarts: [K, L]
+                all_tokens = self.soft_opt.data.argmax(dim=-1)
+
+                # Majority vote per position
+                consensus = torch.zeros(L, dtype=torch.long, device=self.soft_opt.device)
+                for pos in range(L):
+                    tokens_at_pos = all_tokens[:, pos]  # [K]
+                    # Count occurrences and pick the most common
+                    counts = torch.bincount(tokens_at_pos, minlength=V)
+                    consensus[pos] = counts.argmax()
+
+                # Find worst restart by current loss
+                all_ids = self.soft_opt.data.argmax(dim=-1)
+                losses = self.compute_discrete_loss_batch(all_ids)
+                self.flop_counter.count_forward(self.total_seq_len, batch_size=K)
+                worst_k = losses.argmax().item()
+
+                # Set worst restart to one-hot consensus
+                self.soft_opt.data[worst_k] = 0.0
+                for pos in range(L):
+                    self.soft_opt.data[worst_k, pos, consensus[pos]] = 10.0
+
+                # Reset momentum for the modified restart
+                if self.optimizer.state:
+                    for group in self.optimizer.param_groups:
+                        for p in group["params"]:
+                            if p in self.optimizer.state:
+                                buf = self.optimizer.state[p].get("momentum_buffer")
+                                if buf is not None:
+                                    buf[worst_k] = 0.0
+
+                # Reset stagnation for this restart
+                self._stagnant_count[worst_k] = 0
+                self._best_per_restart[worst_k] = float("inf")
+
+                self.log("voted", 1)
+
+        return result

claudini/methods/claude_random/v111/__init__.py

@@ -0,0 +1,3 @@
+from claudini.methods.claude_random.v111.optimizer import ClaudeV111Optimizer
+
+__all__ = ["ClaudeV111Optimizer"]

claudini/methods/claude_random/v111/optimizer.py

@@ -0,0 +1,70 @@
+"""Claude v111: Restart pruning. Every 500 steps, clone best restart into worst if worst is >3x best loss."""
+
+import logging
+
+import torch
+from transformers import PreTrainedModel, PreTrainedTokenizerBase
+
+from claudini.methods.claude_random.v86 import ClaudeV86Optimizer
+
+logger = logging.getLogger("claudini")
+
+
+class ClaudeV111Optimizer(ClaudeV86Optimizer):
+    method_name = "claude_v111"
+
+    def __init__(
+        self,
+        model: PreTrainedModel,
+        tokenizer: PreTrainedTokenizerBase,
+        optim_length: int = 20,
+        lr: float = 10.0,
+        momentum: float = 0.99,
+        ema_alpha: float = 0.01,
+        num_starts: int = 8,
+        lsgm_gamma: float = 0.70,
+        seed: int | None = None,
+        allow_non_ascii: bool = False,
+    ):
+        super().__init__(
+            model, tokenizer, optim_length, lr, momentum, ema_alpha, num_starts, lsgm_gamma, seed, allow_non_ascii
+        )
+        self.patience = 50
+        self.prune_interval = 500
+
+    def step(self, step_num: int) -> tuple[float, float | None, str]:
+        result = super().step(step_num)
+
+        # Restart pruning every prune_interval steps
+        if step_num > 0 and step_num % self.prune_interval == 0:
+            with torch.no_grad():
+                # Get current losses
+                all_ids = self.soft_opt.data.argmax(dim=-1)
+                losses = self.compute_discrete_loss_batch(all_ids)
+                self.flop_counter.count_forward(self.total_seq_len, batch_size=self.num_starts)
+
+                best_k = losses.argmin().item()
+                worst_k = losses.argmax().item()
+                best_loss = losses[best_k].item()
+                worst_loss = losses[worst_k].item()
+
+                # Clone best into worst if worst is >3x best
+                if best_k != worst_k and worst_loss > 3.0 * best_loss:
+                    self.soft_opt.data[worst_k] = self.soft_opt.data[best_k].clone()
+
+                    # Reset momentum for cloned restart
+                    if self.optimizer.state:
+                        for group in self.optimizer.param_groups:
+                            for p in group["params"]:
+                                if p in self.optimizer.state:
+                                    buf = self.optimizer.state[p].get("momentum_buffer")
+                                    if buf is not None:
+                                        buf[worst_k] = buf[best_k].clone()
+
+                    # Reset stagnation for cloned restart
+                    self._stagnant_count[worst_k] = 0
+                    self._best_per_restart[worst_k] = losses[best_k].clone()
+
+                    self.log("pruned", 1)
+
+        return result

claudini/methods/claude_random/v112/__init__.py

@@ -0,0 +1,3 @@
+from claudini.methods.claude_random.v112.optimizer import ClaudeV112Optimizer
+
+__all__ = ["ClaudeV112Optimizer"]

claudini/methods/claude_random/v112/optimizer.py

@@ -0,0 +1,31 @@
+"""Claude v112: patience=50 + lsgm_gamma=0.60. Test lower gamma with perturbation."""
+
+import logging
+
+from transformers import PreTrainedModel, PreTrainedTokenizerBase
+
+from claudini.methods.claude_random.v86 import ClaudeV86Optimizer
+
+logger = logging.getLogger("claudini")
+
+
+class ClaudeV112Optimizer(ClaudeV86Optimizer):
+    method_name = "claude_v112"
+
+    def __init__(
+        self,
+        model: PreTrainedModel,
+        tokenizer: PreTrainedTokenizerBase,
+        optim_length: int = 20,
+        lr: float = 10.0,
+        momentum: float = 0.99,
+        ema_alpha: float = 0.01,
+        num_starts: int = 8,
+        lsgm_gamma: float = 0.60,
+        seed: int | None = None,
+        allow_non_ascii: bool = False,
+    ):
+        super().__init__(
+            model, tokenizer, optim_length, lr, momentum, ema_alpha, num_starts, lsgm_gamma, seed, allow_non_ascii
+        )
+        self.patience = 50

claudini/methods/claude_random/v113/__init__.py

@@ -0,0 +1,3 @@
+from claudini.methods.claude_random.v113.optimizer import ClaudeV113Optimizer
+
+__all__ = ["ClaudeV113Optimizer"]

claudini/methods/claude_random/v113/optimizer.py

@@ -0,0 +1,100 @@
+"""Claude v113: Mixed-scale perturbation. Perturb 2 high-entropy + 2 random positions for balanced exploration."""
+
+import logging
+
+import torch
+from transformers import PreTrainedModel, PreTrainedTokenizerBase
+
+from claudini.methods.claude_random.v86 import ClaudeV86Optimizer
+
+logger = logging.getLogger("claudini")
+
+
+class ClaudeV113Optimizer(ClaudeV86Optimizer):
+    method_name = "claude_v113"
+
+    def __init__(
+        self,
+        model: PreTrainedModel,
+        tokenizer: PreTrainedTokenizerBase,
+        optim_length: int = 20,
+        lr: float = 10.0,
+        momentum: float = 0.99,
+        ema_alpha: float = 0.01,
+        num_starts: int = 8,
+        lsgm_gamma: float = 0.70,
+        seed: int | None = None,
+        allow_non_ascii: bool = False,
+    ):
+        super().__init__(
+            model, tokenizer, optim_length, lr, momentum, ema_alpha, num_starts, lsgm_gamma, seed, allow_non_ascii
+        )
+        self.patience = 50
+
+    def step(self, step_num: int) -> tuple[float, float | None, str]:
+        # Run v26 step (skip v86's step to implement our own perturbation logic)
+        result = super(ClaudeV86Optimizer, self).step(step_num)
+
+        with torch.no_grad():
+            # Get per-restart discrete losses
+            all_ids = self.soft_opt.data.argmax(dim=-1)
+            losses = self.compute_discrete_loss_batch(all_ids)
+            self.flop_counter.count_forward(self.total_seq_len, batch_size=self.num_starts)
+
+            # Track stagnation
+            improved = losses < self._best_per_restart
+            self._best_per_restart = torch.where(improved, losses, self._best_per_restart)
+            self._stagnant_count = torch.where(
+                improved, torch.zeros_like(self._stagnant_count), self._stagnant_count + 1
+            )
+
+            # Perturb stagnant restarts with mixed-scale selection
+            stagnant_mask = self._stagnant_count >= self.patience
+            if stagnant_mask.any():
+                n_stagnant = stagnant_mask.sum().item()
+                L, V = self.soft_opt.data.shape[1], self.soft_opt.data.shape[2]
+
+                for k in range(self.num_starts):
+                    if stagnant_mask[k]:
+                        # Compute entropy per position for this restart
+                        logits_k = self.soft_opt.data[k]  # [L, V]
+                        probs = torch.softmax(logits_k, dim=-1)
+                        entropy = -(probs * (probs + 1e-10).log()).sum(dim=-1)  # [L]
+
+                        # Pick 2 highest-entropy positions (most uncertain — fine-grained escape)
+                        n_entropy = min(2, L)
+                        entropy_positions = entropy.topk(n_entropy).indices
+
+                        # Pick 2 random positions (broad exploration), avoiding entropy positions
+                        remaining_mask = torch.ones(L, dtype=torch.bool, device=self.soft_opt.device)
+                        remaining_mask[entropy_positions] = False
+                        remaining_indices = remaining_mask.nonzero(as_tuple=True)[0]
+
+                        n_random = min(2, len(remaining_indices))
+                        if n_random > 0:
+                            random_positions = remaining_indices[
+                                torch.randperm(len(remaining_indices), device=self.soft_opt.device)[:n_random]
+                            ]
+                            positions = torch.cat([entropy_positions, random_positions])
+                        else:
+                            # If L <= 2, just use entropy positions
+                            positions = entropy_positions
+
+                        # Randomize selected positions
+                        self.soft_opt.data[k, positions] = 0.0
+                        rand_tokens = torch.randint(0, V, (len(positions),), device=self.soft_opt.device)
+                        self.soft_opt.data[k, positions, rand_tokens] = 10.0
+
+                # Reset stagnation counter and momentum for perturbed restarts
+                self._stagnant_count[stagnant_mask] = 0
+                if self.optimizer.state:
+                    for group in self.optimizer.param_groups:
+                        for p in group["params"]:
+                            if p in self.optimizer.state:
+                                buf = self.optimizer.state[p].get("momentum_buffer")
+                                if buf is not None:
+                                    buf[stagnant_mask] = 0.0
+
+                self.log("perturbed", n_stagnant)
+
+        return result

claudini/methods/claude_random/v114/__init__.py

@@ -0,0 +1,3 @@
+from claudini.methods.claude_random.v114.optimizer import ClaudeV114Optimizer
+
+__all__ = ["ClaudeV114Optimizer"]

claudini/methods/claude_random/v114/optimizer.py

@@ -0,0 +1,31 @@
+"""Claude v114: K=10 restarts with patience=50. More restarts with faster perturbation trigger."""
+
+import logging
+
+from transformers import PreTrainedModel, PreTrainedTokenizerBase
+
+from claudini.methods.claude_random.v86 import ClaudeV86Optimizer
+
+logger = logging.getLogger("claudini")
+
+
+class ClaudeV114Optimizer(ClaudeV86Optimizer):
+    method_name = "claude_v114"
+
+    def __init__(
+        self,
+        model: PreTrainedModel,
+        tokenizer: PreTrainedTokenizerBase,
+        optim_length: int = 20,
+        lr: float = 10.0,
+        momentum: float = 0.99,
+        ema_alpha: float = 0.01,
+        num_starts: int = 10,
+        lsgm_gamma: float = 0.70,
+        seed: int | None = None,
+        allow_non_ascii: bool = False,
+    ):
+        super().__init__(
+            model, tokenizer, optim_length, lr, momentum, ema_alpha, num_starts, lsgm_gamma, seed, allow_non_ascii
+        )
+        self.patience = 50

claudini/methods/claude_random/v115/__init__.py

@@ -0,0 +1,3 @@
+from claudini.methods.claude_random.v115.optimizer import ClaudeV115Optimizer
+
+__all__ = ["ClaudeV115Optimizer"]

claudini/methods/claude_random/v115/optimizer.py

@@ -0,0 +1,31 @@
+"""Claude v115: K=12 restarts with patience=50. More restarts with faster perturbation trigger."""
+
+import logging
+
+from transformers import PreTrainedModel, PreTrainedTokenizerBase
+
+from claudini.methods.claude_random.v86 import ClaudeV86Optimizer
+
+logger = logging.getLogger("claudini")
+
+
+class ClaudeV115Optimizer(ClaudeV86Optimizer):
+    method_name = "claude_v115"
+
+    def __init__(
+        self,
+        model: PreTrainedModel,
+        tokenizer: PreTrainedTokenizerBase,
+        optim_length: int = 20,
+        lr: float = 10.0,
+        momentum: float = 0.99,
+        ema_alpha: float = 0.01,
+        num_starts: int = 12,
+        lsgm_gamma: float = 0.70,
+        seed: int | None = None,
+        allow_non_ascii: bool = False,
+    ):
+        super().__init__(
+            model, tokenizer, optim_length, lr, momentum, ema_alpha, num_starts, lsgm_gamma, seed, allow_non_ascii
+        )
+        self.patience = 50

claudini/methods/claude_random/v116/__init__.py

@@ -0,0 +1,3 @@
+from claudini.methods.claude_random.v116.optimizer import ClaudeV116Optimizer
+
+__all__ = ["ClaudeV116Optimizer"]

claudini/methods/claude_random/v116/optimizer.py

@@ -0,0 +1,27 @@
+"""Claude v116: patience=40 K=8 γ=0.70. Fine-tuning patience around 50."""
+
+from transformers import PreTrainedModel, PreTrainedTokenizerBase
+
+from claudini.methods.claude_random.v86 import ClaudeV86Optimizer
+
+
+class ClaudeV116Optimizer(ClaudeV86Optimizer):
+    method_name = "claude_v116"
+
+    def __init__(
+        self,
+        model: PreTrainedModel,
+        tokenizer: PreTrainedTokenizerBase,
+        optim_length: int = 20,
+        lr: float = 10.0,
+        momentum: float = 0.99,
+        ema_alpha: float = 0.01,
+        num_starts: int = 8,
+        lsgm_gamma: float = 0.70,
+        seed: int | None = None,
+        allow_non_ascii: bool = False,
+    ):
+        super().__init__(
+            model, tokenizer, optim_length, lr, momentum, ema_alpha, num_starts, lsgm_gamma, seed, allow_non_ascii
+        )
+        self.patience = 40

claudini/methods/claude_random/v117/__init__.py

@@ -0,0 +1,3 @@
+from claudini.methods.claude_random.v117.optimizer import ClaudeV117Optimizer
+
+__all__ = ["ClaudeV117Optimizer"]

claudini/methods/claude_random/v117/optimizer.py

@@ -0,0 +1,27 @@
+"""Claude v117: patience=45 K=8 γ=0.70. Fine-tuning patience around 50."""
+
+from transformers import PreTrainedModel, PreTrainedTokenizerBase
+
+from claudini.methods.claude_random.v86 import ClaudeV86Optimizer
+
+
+class ClaudeV117Optimizer(ClaudeV86Optimizer):
+    method_name = "claude_v117"
+
+    def __init__(
+        self,
+        model: PreTrainedModel,
+        tokenizer: PreTrainedTokenizerBase,
+        optim_length: int = 20,
+        lr: float = 10.0,
+        momentum: float = 0.99,
+        ema_alpha: float = 0.01,
+        num_starts: int = 8,
+        lsgm_gamma: float = 0.70,
+        seed: int | None = None,
+        allow_non_ascii: bool = False,
+    ):
+        super().__init__(
+            model, tokenizer, optim_length, lr, momentum, ema_alpha, num_starts, lsgm_gamma, seed, allow_non_ascii
+        )
+        self.patience = 45

claudini/methods/claude_random/v118/__init__.py

@@ -0,0 +1,3 @@
+from claudini.methods.claude_random.v118.optimizer import ClaudeV118Optimizer
+
+__all__ = ["ClaudeV118Optimizer"]

claudini/methods/claude_random/v118/optimizer.py

@@ -0,0 +1,72 @@
+"""Claude v118: Soft perturbation (Gaussian noise σ=2.0 to all positions). Gentler than hard random reset."""
+
+import logging
+
+import torch
+from transformers import PreTrainedModel, PreTrainedTokenizerBase
+
+from claudini.methods.claude_random.v86 import ClaudeV86Optimizer
+
+logger = logging.getLogger("claudini")
+
+
+class ClaudeV118Optimizer(ClaudeV86Optimizer):
+    method_name = "claude_v118"
+
+    def __init__(
+        self,
+        model: PreTrainedModel,
+        tokenizer: PreTrainedTokenizerBase,
+        optim_length: int = 20,
+        lr: float = 10.0,
+        momentum: float = 0.99,
+        ema_alpha: float = 0.01,
+        num_starts: int = 8,
+        lsgm_gamma: float = 0.70,
+        seed: int | None = None,
+        allow_non_ascii: bool = False,
+    ):
+        super().__init__(
+            model, tokenizer, optim_length, lr, momentum, ema_alpha, num_starts, lsgm_gamma, seed, allow_non_ascii
+        )
+
+    def step(self, step_num: int) -> tuple[float, float | None, str]:
+        # Run v26 step (skip v86's perturbation logic entirely)
+        result = super(ClaudeV86Optimizer, self).step(step_num)
+
+        with torch.no_grad():
+            # Get per-restart discrete losses
+            all_ids = self.soft_opt.data.argmax(dim=-1)
+            losses = self.compute_discrete_loss_batch(all_ids)
+            self.flop_counter.count_forward(self.total_seq_len, batch_size=self.num_starts)
+
+            # Track stagnation
+            improved = losses < self._best_per_restart
+            self._best_per_restart = torch.where(improved, losses, self._best_per_restart)
+            self._stagnant_count = torch.where(
+                improved, torch.zeros_like(self._stagnant_count), self._stagnant_count + 1
+            )
+
+            # Soft perturbation: add Gaussian noise to ALL positions of stagnant restarts
+            stagnant_mask = self._stagnant_count >= self.patience
+            if stagnant_mask.any():
+                n_stagnant = stagnant_mask.sum().item()
+
+                for k in range(self.num_starts):
+                    if stagnant_mask[k]:
+                        noise = torch.randn_like(self.soft_opt.data[k]) * 2.0
+                        self.soft_opt.data[k] += noise
+
+                # Reset stagnation counter and momentum for perturbed restarts
+                self._stagnant_count[stagnant_mask] = 0
+                if self.optimizer.state:
+                    for group in self.optimizer.param_groups:
+                        for p in group["params"]:
+                            if p in self.optimizer.state:
+                                buf = self.optimizer.state[p].get("momentum_buffer")
+                                if buf is not None:
+                                    buf[stagnant_mask] = 0.0
+
+                self.log("perturbed", n_stagnant)
+
+        return result

claudini/methods/claude_random/v119/__init__.py

@@ -0,0 +1,3 @@
+from claudini.methods.claude_random.v119.optimizer import ClaudeV119Optimizer
+
+__all__ = ["ClaudeV119Optimizer"]

claudini/methods/claude_random/v119/optimizer.py

@@ -0,0 +1,27 @@
+"""Claude v119: momentum=0.98 + patience=50. Faster momentum adaptation."""
+
+from transformers import PreTrainedModel, PreTrainedTokenizerBase
+
+from claudini.methods.claude_random.v86 import ClaudeV86Optimizer
+
+
+class ClaudeV119Optimizer(ClaudeV86Optimizer):
+    method_name = "claude_v119"
+
+    def __init__(
+        self,
+        model: PreTrainedModel,
+        tokenizer: PreTrainedTokenizerBase,
+        optim_length: int = 20,
+        lr: float = 10.0,
+        momentum: float = 0.98,
+        ema_alpha: float = 0.01,
+        num_starts: int = 8,
+        lsgm_gamma: float = 0.70,
+        seed: int | None = None,
+        allow_non_ascii: bool = False,
+    ):
+        super().__init__(
+            model, tokenizer, optim_length, lr, momentum, ema_alpha, num_starts, lsgm_gamma, seed, allow_non_ascii
+        )
+        self.patience = 50

claudini/methods/claude_random/v12/__init__.py

@@ -0,0 +1,3 @@
+from .optimizer import ClaudeV12Optimizer
+
+__all__ = ["ClaudeV12Optimizer"]

claudini/methods/claude_random/v12/optimizer.py

@@ -0,0 +1,40 @@
+"""
+Claude v12 optimizer: ADC + LSGM, K=32 restarts.
+
+Base: v6 (ADC + LSGM gamma=0.5, K=16) — avg 0.80.
+Change: Double restarts to K=32 (was 16).
+
+Motivation: more restarts = more basins explored = higher chance at least
+one restart finds a near-zero solution. Trade-off: ~1008 steps (vs 2016
+for K=16). v6 showed some seeds stuck at 1.34-1.98 — more restarts might
+reduce variance by exploring more starting points.
+"""
+
+import logging
+
+from transformers import PreTrainedModel, PreTrainedTokenizerBase
+
+from claudini.methods.claude_random.v6 import ClaudeV6Optimizer
+
+logger = logging.getLogger("claudini")
+
+
+class ClaudeV12Optimizer(ClaudeV6Optimizer):
+    method_name = "claude_v12"
+
+    def __init__(
+        self,
+        model: PreTrainedModel,
+        tokenizer: PreTrainedTokenizerBase,
+        optim_length: int = 20,
+        lr: float = 10.0,
+        momentum: float = 0.99,
+        ema_alpha: float = 0.01,
+        num_starts: int = 32,
+        lsgm_gamma: float = 0.5,
+        seed: int | None = None,
+        allow_non_ascii: bool = False,
+    ):
+        super().__init__(
+            model, tokenizer, optim_length, lr, momentum, ema_alpha, num_starts, lsgm_gamma, seed, allow_non_ascii
+        )

claudini/methods/claude_random/v120/__init__.py

@@ -0,0 +1,3 @@
+from claudini.methods.claude_random.v120.optimizer import ClaudeV120Optimizer
+
+__all__ = ["ClaudeV120Optimizer"]

claudini/methods/claude_random/v120/optimizer.py

@@ -0,0 +1,37 @@
+"""Claude v120: LR cycling (8↔12 period=200) + patience=50. Explore different lr regimes."""
+
+import math
+
+from transformers import PreTrainedModel, PreTrainedTokenizerBase
+
+from claudini.methods.claude_random.v86 import ClaudeV86Optimizer
+
+
+class ClaudeV120Optimizer(ClaudeV86Optimizer):
+    method_name = "claude_v120"
+
+    def __init__(
+        self,
+        model: PreTrainedModel,
+        tokenizer: PreTrainedTokenizerBase,
+        optim_length: int = 20,
+        lr: float = 10.0,
+        momentum: float = 0.99,
+        ema_alpha: float = 0.01,
+        num_starts: int = 8,
+        lsgm_gamma: float = 0.70,
+        seed: int | None = None,
+        allow_non_ascii: bool = False,
+    ):
+        super().__init__(
+            model, tokenizer, optim_length, lr, momentum, ema_alpha, num_starts, lsgm_gamma, seed, allow_non_ascii
+        )
+        self.patience = 50
+
+    def step(self, step_num: int) -> tuple[float, float | None, str]:
+        # Adjust learning rate before calling parent step
+        new_lr = 10 + 2 * math.sin(2 * math.pi * step_num / 200)
+        for group in self.optimizer.param_groups:
+            group["lr"] = new_lr
+
+        return super().step(step_num)

claudini/methods/claude_random/v121/__init__.py

@@ -0,0 +1,3 @@
+from claudini.methods.claude_random.v121.optimizer import ClaudeV121Optimizer
+
+__all__ = ["ClaudeV121Optimizer"]

claudini/methods/claude_random/v121/optimizer.py

@@ -0,0 +1,41 @@
+"""Claude v121: Periodic momentum reset every 200 steps. Re-evaluate gradient landscape without position perturbation."""
+
+import torch
+from transformers import PreTrainedModel, PreTrainedTokenizerBase
+
+from claudini.methods.claude_random.v26 import ClaudeV26Optimizer
+
+
+class ClaudeV121Optimizer(ClaudeV26Optimizer):
+    method_name = "claude_v121"
+
+    def __init__(
+        self,
+        model: PreTrainedModel,
+        tokenizer: PreTrainedTokenizerBase,
+        optim_length: int = 20,
+        lr: float = 10.0,
+        momentum: float = 0.99,
+        ema_alpha: float = 0.01,
+        num_starts: int = 8,
+        lsgm_gamma: float = 0.70,
+        seed: int | None = None,
+        allow_non_ascii: bool = False,
+    ):
+        super().__init__(
+            model, tokenizer, optim_length, lr, momentum, ema_alpha, num_starts, lsgm_gamma, seed, allow_non_ascii
+        )
+
+    def step(self, step_num: int) -> tuple[float, float | None, str]:
+        # Reset momentum buffer every 200 steps
+        if step_num > 0 and step_num % 200 == 0:
+            with torch.no_grad():
+                if self.optimizer.state:
+                    for group in self.optimizer.param_groups:
+                        for p in group["params"]:
+                            if p in self.optimizer.state:
+                                buf = self.optimizer.state[p].get("momentum_buffer")
+                                if buf is not None:
+                                    buf.zero_()
+
+        return super().step(step_num)

claudini/methods/claude_random/v122/__init__.py

@@ -0,0 +1,3 @@
+from claudini.methods.claude_random.v122.optimizer import ClaudeV122Optimizer
+
+__all__ = ["ClaudeV122Optimizer"]

claudini/methods/claude_random/v122/optimizer.py

@@ -0,0 +1,27 @@
+"""Claude v122: Restore-best + patience=40. Combining restore with shorter patience."""
+
+from transformers import PreTrainedModel, PreTrainedTokenizerBase
+
+from claudini.methods.claude_random.v90 import ClaudeV90Optimizer
+
+
+class ClaudeV122Optimizer(ClaudeV90Optimizer):
+    method_name = "claude_v122"
+
+    def __init__(
+        self,
+        model: PreTrainedModel,
+        tokenizer: PreTrainedTokenizerBase,
+        optim_length: int = 20,
+        lr: float = 10.0,
+        momentum: float = 0.99,
+        ema_alpha: float = 0.01,
+        num_starts: int = 8,
+        lsgm_gamma: float = 0.70,
+        seed: int | None = None,
+        allow_non_ascii: bool = False,
+    ):
+        super().__init__(
+            model, tokenizer, optim_length, lr, momentum, ema_alpha, num_starts, lsgm_gamma, seed, allow_non_ascii
+        )
+        self.patience = 40

claudini/methods/claude_random/v123/__init__.py

@@ -0,0 +1,3 @@
+from claudini.methods.claude_random.v123.optimizer import ClaudeV123Optimizer
+
+__all__ = ["ClaudeV123Optimizer"]

claudini/methods/claude_random/v123/optimizer.py

@@ -0,0 +1,34 @@
+"""Claude v123: Annealing perturbation. Patience 150→30, n_perturb 2→6 over training."""
+
+from transformers import PreTrainedModel, PreTrainedTokenizerBase
+
+from claudini.methods.claude_random.v86 import ClaudeV86Optimizer
+
+
+class ClaudeV123Optimizer(ClaudeV86Optimizer):
+    method_name = "claude_v123"
+
+    def __init__(
+        self,
+        model: PreTrainedModel,
+        tokenizer: PreTrainedTokenizerBase,
+        optim_length: int = 20,
+        lr: float = 10.0,
+        momentum: float = 0.99,
+        ema_alpha: float = 0.01,
+        num_starts: int = 8,
+        lsgm_gamma: float = 0.70,
+        seed: int | None = None,
+        allow_non_ascii: bool = False,
+    ):
+        super().__init__(
+            model, tokenizer, optim_length, lr, momentum, ema_alpha, num_starts, lsgm_gamma, seed, allow_non_ascii
+        )
+
+    def step(self, step_num: int) -> tuple[float, float | None, str]:
+        # Compute dynamic patience and n_perturb based on progress
+        progress = min(step_num / 4000, 1.0)
+        self.patience = int(150 - 120 * progress)  # 150 -> 30
+        self.n_perturb = int(2 + 4 * progress)  # 2 -> 6
+
+        return super().step(step_num)

claudini/methods/claude_random/v124/__init__.py

@@ -0,0 +1,3 @@
+from claudini.methods.claude_random.v124.optimizer import ClaudeV124Optimizer
+
+__all__ = ["ClaudeV124Optimizer"]

claudini/methods/claude_random/v124/optimizer.py

@@ -0,0 +1,27 @@
+"""Claude v124: gamma=0.68 + patience=50. Test slightly lower gamma."""
+
+from transformers import PreTrainedModel, PreTrainedTokenizerBase
+
+from claudini.methods.claude_random.v86 import ClaudeV86Optimizer
+
+
+class ClaudeV124Optimizer(ClaudeV86Optimizer):
+    method_name = "claude_v124"
+
+    def __init__(
+        self,
+        model: PreTrainedModel,
+        tokenizer: PreTrainedTokenizerBase,
+        optim_length: int = 20,
+        lr: float = 10.0,
+        momentum: float = 0.99,
+        ema_alpha: float = 0.01,
+        num_starts: int = 8,
+        lsgm_gamma: float = 0.68,
+        seed: int | None = None,
+        allow_non_ascii: bool = False,
+    ):
+        super().__init__(
+            model, tokenizer, optim_length, lr, momentum, ema_alpha, num_starts, lsgm_gamma, seed, allow_non_ascii
+        )
+        self.patience = 50

claudini/methods/claude_random/v13/__init__.py

@@ -0,0 +1,3 @@
+from .optimizer import ClaudeV13Optimizer
+
+__all__ = ["ClaudeV13Optimizer"]

claudini/methods/claude_random/v13/optimizer.py

@@ -0,0 +1,40 @@
+"""
+Claude v13 optimizer: ADC + LSGM gamma=0.7.
+
+Base: v6 (ADC + LSGM gamma=0.5, K=16) — avg 0.80.
+Change: Milder LSGM scaling (gamma=0.7 vs 0.5).
+
+Motivation: gamma=0.3 was too aggressive (v10: 11.25). gamma=0.5 works
+great (v6: 0.80). Maybe gamma=0.7 is even better? Less gradient damping
+preserves more of the original gradient signal while still amplifying
+skip connections. The optimal gamma might be between 0.5 and 1.0.
+"""
+
+import logging
+
+from transformers import PreTrainedModel, PreTrainedTokenizerBase
+
+from claudini.methods.claude_random.v6 import ClaudeV6Optimizer
+
+logger = logging.getLogger("claudini")
+
+
+class ClaudeV13Optimizer(ClaudeV6Optimizer):
+    method_name = "claude_v13"
+
+    def __init__(
+        self,
+        model: PreTrainedModel,
+        tokenizer: PreTrainedTokenizerBase,
+        optim_length: int = 20,
+        lr: float = 10.0,
+        momentum: float = 0.99,
+        ema_alpha: float = 0.01,
+        num_starts: int = 16,
+        lsgm_gamma: float = 0.7,
+        seed: int | None = None,
+        allow_non_ascii: bool = False,
+    ):
+        super().__init__(
+            model, tokenizer, optim_length, lr, momentum, ema_alpha, num_starts, lsgm_gamma, seed, allow_non_ascii
+        )

claudini/methods/claude_random/v14/__init__.py

@@ -0,0 +1,3 @@
+from .optimizer import ClaudeV14Optimizer
+
+__all__ = ["ClaudeV14Optimizer"]

claudini/methods/claude_random/v14/optimizer.py

@@ -0,0 +1,34 @@
+"""
+Claude v14 optimizer: ADC + LSGM gamma=0.6.
+
+Gamma sweep: 0.3=11.25, 0.5=0.80, 0.7=0.44. Trying 0.6 to narrow the optimum.
+"""
+
+import logging
+
+from transformers import PreTrainedModel, PreTrainedTokenizerBase
+
+from claudini.methods.claude_random.v6 import ClaudeV6Optimizer
+
+logger = logging.getLogger("claudini")
+
+
+class ClaudeV14Optimizer(ClaudeV6Optimizer):
+    method_name = "claude_v14"
+
+    def __init__(
+        self,
+        model: PreTrainedModel,
+        tokenizer: PreTrainedTokenizerBase,
+        optim_length: int = 20,
+        lr: float = 10.0,
+        momentum: float = 0.99,
+        ema_alpha: float = 0.01,
+        num_starts: int = 16,
+        lsgm_gamma: float = 0.6,
+        seed: int | None = None,
+        allow_non_ascii: bool = False,
+    ):
+        super().__init__(
+            model, tokenizer, optim_length, lr, momentum, ema_alpha, num_starts, lsgm_gamma, seed, allow_non_ascii
+        )

claudini/methods/claude_random/v15/__init__.py

@@ -0,0 +1,3 @@
+from .optimizer import ClaudeV15Optimizer
+
+__all__ = ["ClaudeV15Optimizer"]

claudini/methods/claude_random/v15/optimizer.py

@@ -0,0 +1,34 @@
+"""
+Claude v15 optimizer: ADC + LSGM gamma=0.8.
+
+Gamma sweep: 0.3=11.25, 0.5=0.80, 0.7=0.44. Trying 0.8 to check if even milder is better.
+"""
+
+import logging
+
+from transformers import PreTrainedModel, PreTrainedTokenizerBase
+
+from claudini.methods.claude_random.v6 import ClaudeV6Optimizer
+
+logger = logging.getLogger("claudini")
+
+
+class ClaudeV15Optimizer(ClaudeV6Optimizer):
+    method_name = "claude_v15"
+
+    def __init__(
+        self,
+        model: PreTrainedModel,
+        tokenizer: PreTrainedTokenizerBase,
+        optim_length: int = 20,
+        lr: float = 10.0,
+        momentum: float = 0.99,
+        ema_alpha: float = 0.01,
+        num_starts: int = 16,
+        lsgm_gamma: float = 0.8,
+        seed: int | None = None,
+        allow_non_ascii: bool = False,
+    ):
+        super().__init__(
+            model, tokenizer, optim_length, lr, momentum, ema_alpha, num_starts, lsgm_gamma, seed, allow_non_ascii
+        )

claudini/methods/claude_random/v16/__init__.py

@@ -0,0 +1,3 @@
+from .optimizer import ClaudeV16Optimizer
+
+__all__ = ["ClaudeV16Optimizer"]

claudini/methods/claude_random/v16/optimizer.py

@@ -0,0 +1,35 @@
+"""
+Claude v16 optimizer: ADC + LSGM gamma=0.6, lr=20.0 (2× default).
+
+Gamma sweep found 0.6 optimal. Now testing if higher lr helps convergence.
+Default lr=10.0 (effective 160 with K=16). This uses lr=20.0 (effective 320).
+"""
+
+import logging
+
+from transformers import PreTrainedModel, PreTrainedTokenizerBase
+
+from claudini.methods.claude_random.v6 import ClaudeV6Optimizer
+
+logger = logging.getLogger("claudini")
+
+
+class ClaudeV16Optimizer(ClaudeV6Optimizer):
+    method_name = "claude_v16"
+
+    def __init__(
+        self,
+        model: PreTrainedModel,
+        tokenizer: PreTrainedTokenizerBase,
+        optim_length: int = 20,
+        lr: float = 20.0,
+        momentum: float = 0.99,
+        ema_alpha: float = 0.01,
+        num_starts: int = 16,
+        lsgm_gamma: float = 0.6,
+        seed: int | None = None,
+        allow_non_ascii: bool = False,
+    ):
+        super().__init__(
+            model, tokenizer, optim_length, lr, momentum, ema_alpha, num_starts, lsgm_gamma, seed, allow_non_ascii
+        )

claudini/methods/claude_random/v17/__init__.py

@@ -0,0 +1,3 @@
+from .optimizer import ClaudeV17Optimizer
+
+__all__ = ["ClaudeV17Optimizer"]

claudini/methods/claude_random/v17/optimizer.py

@@ -0,0 +1,35 @@
+"""
+Claude v17 optimizer: ADC + LSGM gamma=0.6, lr=5.0 (0.5× default).
+
+Gamma sweep found 0.6 optimal. Testing if lower lr (more cautious updates) improves
+consistency. Default lr=10.0 (effective 160 with K=16). This uses lr=5.0 (effective 80).
+"""
+
+import logging
+
+from transformers import PreTrainedModel, PreTrainedTokenizerBase
+
+from claudini.methods.claude_random.v6 import ClaudeV6Optimizer
+
+logger = logging.getLogger("claudini")
+
+
+class ClaudeV17Optimizer(ClaudeV6Optimizer):
+    method_name = "claude_v17"
+
+    def __init__(
+        self,
+        model: PreTrainedModel,
+        tokenizer: PreTrainedTokenizerBase,
+        optim_length: int = 20,
+        lr: float = 5.0,
+        momentum: float = 0.99,
+        ema_alpha: float = 0.01,
+        num_starts: int = 16,
+        lsgm_gamma: float = 0.6,
+        seed: int | None = None,
+        allow_non_ascii: bool = False,
+    ):
+        super().__init__(
+            model, tokenizer, optim_length, lr, momentum, ema_alpha, num_starts, lsgm_gamma, seed, allow_non_ascii
+        )

claudini/methods/claude_random/v18/__init__.py

@@ -0,0 +1,3 @@
+from .optimizer import ClaudeV18Optimizer
+
+__all__ = ["ClaudeV18Optimizer"]

claudini/methods/claude_random/v18/optimizer.py

@@ -0,0 +1,47 @@
+"""
+Claude v18 optimizer: ADC + LSGM gamma=0.6 + LILA.
+
+v11 (LSGM 0.5 + LILA) got 1.50 — high variance but 4/5 seeds were incredible.
+v14 (LSGM 0.6) got 0.42 — best pure LSGM gamma.
+Combining optimal gamma=0.6 with LILA to see if LILA's upside + better gamma = improvement.
+"""
+
+import logging
+
+from transformers import PreTrainedModel, PreTrainedTokenizerBase
+
+from claudini.methods.claude_random.v11 import ClaudeV11Optimizer
+
+logger = logging.getLogger("claudini")
+
+
+class ClaudeV18Optimizer(ClaudeV11Optimizer):
+    method_name = "claude_v18"
+
+    def __init__(
+        self,
+        model: PreTrainedModel,
+        tokenizer: PreTrainedTokenizerBase,
+        optim_length: int = 20,
+        lr: float = 10.0,
+        momentum: float = 0.99,
+        ema_alpha: float = 0.01,
+        num_starts: int = 16,
+        lsgm_gamma: float = 0.6,
+        lila_layer: int | None = None,
+        seed: int | None = None,
+        allow_non_ascii: bool = False,
+    ):
+        super().__init__(
+            model,
+            tokenizer,
+            optim_length,
+            lr,
+            momentum,
+            ema_alpha,
+            num_starts,
+            lsgm_gamma,
+            lila_layer,
+            seed,
+            allow_non_ascii,
+        )

claudini/methods/claude_random/v19/__init__.py

@@ -0,0 +1,3 @@
+from .optimizer import ClaudeV19Optimizer
+
+__all__ = ["ClaudeV19Optimizer"]

claudini/methods/claude_random/v19/optimizer.py

@@ -0,0 +1,131 @@
+"""
+Claude v19 optimizer: ADC with decoupled K/lr.
+
+Key fix: loss uses sum() instead of mean() over restarts, so lr is independent of K.
+Original ADC: lr_effective = lr * K (to compensate for mean). This ties K and lr together.
+v19: lr_effective = lr (independent). K controls exploration, lr controls step size.
+
+This enables scaling K to 64+ without blowing up the learning rate.
+No LSGM (hurts on Llama-2). Targeting Llama-2 where ADC alone gets 5.33.
+"""
+
+import logging
+
+import torch
+from transformers import PreTrainedModel, PreTrainedTokenizerBase
+
+from claudini.methods.original.adc import ADCOptimizer
+
+logger = logging.getLogger("claudini")
+
+
+class ClaudeV19Optimizer(ADCOptimizer):
+    method_name = "claude_v19"
+
+    def __init__(
+        self,
+        model: PreTrainedModel,
+        tokenizer: PreTrainedTokenizerBase,
+        optim_length: int = 20,
+        lr: float = 10.0,
+        momentum: float = 0.99,
+        ema_alpha: float = 0.01,
+        num_starts: int = 64,
+        seed: int | None = None,
+        allow_non_ascii: bool = False,
+    ):
+        super().__init__(model, tokenizer, optim_length, lr, momentum, ema_alpha, num_starts, seed, allow_non_ascii)
+        # DECOUPLE: override the lr*K scaling from parent
+        self.lr = lr  # NOT lr * num_starts
+
+    def setup(self, prompt: str, target: str) -> None:
+        super().setup(prompt, target)
+        # Re-create optimizer with decoupled lr (parent used lr*K)
+        self.optimizer = torch.optim.SGD(
+            [self.soft_opt],
+            lr=self.lr,
+            momentum=self.momentum,
+        )
+
+    def step(self, step_num: int) -> tuple[float, float | None, str]:
+        """ADC step with sum() loss instead of mean() — decouples K from lr."""
+        K = self.num_starts
+        self.optimizer.zero_grad()
+
+        # 1. Soft embeddings for all K restarts: [K, L, V] @ [V, D] -> [K, L, D]
+        W = self.embedding_layer.weight.detach()
+        soft_embeds = torch.matmul(
+            self.soft_opt.to(torch.float32),
+            W.to(torch.float32),
+        ).to(self.model_dtype)
+
+        # 2. Batched forward: [K, seq_len, D]
+        input_embeds = torch.cat(
+            [
+                self.before_embeds.expand(K, -1, -1),
+                soft_embeds,
+                self.after_embeds.expand(K, -1, -1),
+                self.target_embeds.expand(K, -1, -1),
+            ],
+            dim=1,
+        )
+
+        logits = self.model(inputs_embeds=input_embeds).logits
+        shift = input_embeds.shape[1] - self.target_ids.shape[1]
+        target_len = self.target_ids.shape[1]
+        shift_logits = logits[..., shift - 1 : shift - 1 + target_len, :].contiguous()
+
+        # 3. Per-restart CE loss, SUMMED over K (decoupled from lr)
+        target_expanded = self.target_ids.expand(K, -1)
+        loss_per_token = torch.nn.functional.cross_entropy(
+            shift_logits.view(-1, shift_logits.size(-1)),
+            target_expanded.reshape(-1),
+            reduction="none",
+        )
+        loss_per_restart = loss_per_token.view(K, target_len).mean(dim=1)  # [K] — mean over tokens
+        soft_loss = loss_per_restart.sum()  # SUM over K (not mean!) — decouples lr from K
+        soft_loss_val = float(soft_loss.item() / K)  # Report mean for logging
+
+        # Wrong prediction count per restart for adaptive sparsity
+        with torch.no_grad():
+            preds = shift_logits.argmax(dim=-1)
+            wrong_counts = (preds != target_expanded).float().sum(dim=1)
+
+        soft_loss.backward()
+        self.optimizer.step()
+
+        self.flop_counter.count_forward_backward(self.total_seq_len, batch_size=K)
+
+        with torch.no_grad():
+            # 4. Adaptive sparsity per restart
+            if self.running_wrong is None:
+                self.running_wrong = wrong_counts.clone()
+            else:
+                self.running_wrong += (wrong_counts - self.running_wrong) * self.ema_alpha
+
+            sparsities = (2.0**self.running_wrong).clamp(max=self.vocab_size / 2)
+
+            if self.forbidden_mask is not None:
+                self.soft_opt.data[:, :, self.forbidden_mask] = -1000.0
+
+            pre_sparse = self.soft_opt.data.clone()
+
+            sparse_z = self._make_sparse_batched(self.soft_opt.data, sparsities)
+            self.soft_opt.data.copy_(sparse_z)
+
+            # 6. Discrete eval: argmax per restart
+            all_ids = pre_sparse.argmax(dim=-1)
+            discrete_losses = self.compute_discrete_loss_batch(all_ids)
+            self.flop_counter.count_forward(self.total_seq_len, batch_size=K)
+
+            best_k = discrete_losses.argmin().item()
+            step_best_loss = discrete_losses[best_k].item()
+
+            if step_best_loss < self._global_best_loss:
+                self._global_best_loss = step_best_loss
+                self._global_best_ids = all_ids[best_k].clone()
+
+            self._step_ids = self._global_best_ids
+            optim_str = self.tokenizer.decode(self._global_best_ids)
+
+        return step_best_loss, soft_loss_val, optim_str

claudini/methods/claude_random/v2/__init__.py

@@ -0,0 +1,3 @@
+from .optimizer import ClaudeV2Optimizer
+
+__all__ = ["ClaudeV2Optimizer"]

claudini/methods/claude_random/v2/optimizer.py

@@ -0,0 +1,275 @@
+"""
+Claude v2 optimizer: fewer restarts, wider search.
+
+Changes from v1:
+  - K=2 restarts (was 4) — each gets 2× more candidates
+  - search_width 128→512 (was 64→256) — matches ACG's scale
+  - Keeps LSGM, momentum, best-ever buffer, patience
+
+Rationale: v1 spread candidates too thin across 4 restarts (only 64 each).
+gcg_fast (K=4, sw=128) wins on llama2/gemma but loses to i_gcg on Qwen.
+Hypothesis: fewer restarts + wider search + LSGM = better on Qwen.
+"""
+
+import logging
+
+import torch
+from torch import Tensor
+from transformers import PreTrainedModel, PreTrainedTokenizerBase
+
+from claudini.base import TokenOptimizer
+from claudini.tokens import sample_ids_from_grad
+
+logger = logging.getLogger("claudini")
+
+
+class ClaudeV2Optimizer(TokenOptimizer):
+    method_name = "claude_v2"
+
+    def __init__(
+        self,
+        model: PreTrainedModel,
+        tokenizer: PreTrainedTokenizerBase,
+        optim_length: int = 20,
+        # Multi-restart — reduced from 4 to 2
+        num_starts: int = 2,
+        # ACG-style adaptive schedules — wider range
+        n_replace_max: int = 5,
+        n_replace_min: int = 1,
+        search_width_min: int = 128,
+        search_width_max: int = 512,
+        topk_per_position: int = 256,
+        # LSGM
+        lsgm_gamma: float = 0.5,
+        # Momentum
+        momentum: float = 0.5,
+        # Patience
+        patience: int = 50,
+        n_perturb: int = 3,
+        seed: int | None = None,
+        allow_non_ascii: bool = False,
+    ):
+        super().__init__(model, tokenizer, optim_length, seed, allow_non_ascii)
+        self.num_starts = num_starts
+        self.n_replace_max = n_replace_max
+        self.n_replace_min = n_replace_min
+        self.search_width_min = search_width_min
+        self.search_width_max = search_width_max
+        self.topk_per_position = topk_per_position
+        self.lsgm_gamma = lsgm_gamma
+        self.momentum = momentum
+        self.patience_limit = patience
+        self.n_perturb = n_perturb
+
+        # State (initialized in setup)
+        self.current_ids: Tensor | None = None
+        self.best_ids: Tensor | None = None
+        self.best_losses: list | None = None
+        self._restart_patience: list | None = None
+        self._momentum_buffer: list | None = None
+        self._lsgm_handles: list = []
+        self.max_flops: float | None = None
+
+    def _get_progress(self) -> float:
+        if self.max_flops is None or self.max_flops <= 0:
+            return 0.0
+        return min(1.0, self.flop_counter.total_flops / self.max_flops)
+
+    def _get_n_replace(self) -> int:
+        t = self._get_progress()
+        m = self.n_replace_max + t * (self.n_replace_min - self.n_replace_max)
+        return max(self.n_replace_min, int(round(m)))
+
+    def _get_search_width(self) -> int:
+        t = self._get_progress()
+        B = self.search_width_min + t * (self.search_width_max - self.search_width_min)
+        return max(1, int(round(B)))
+
+    # --- LSGM hooks ---
+
+    def _get_norm_modules(self):
+        norms = []
+        for name, module in self.model.named_modules():
+            if any(
+                p in name
+                for p in [
+                    "input_layernorm",
+                    "post_attention_layernorm",
+                    "pre_feedforward_layernorm",
+                    "post_feedforward_layernorm",
+                    ".ln_1",
+                    ".ln_2",
+                ]
+            ):
+                norms.append(module)
+        return norms
+
+    def _register_lsgm_hooks(self) -> list:
+        handles = []
+        gamma = self.lsgm_gamma
+        for module in self._get_norm_modules():
+
+            def hook(m, grad_input, grad_output, _gamma=gamma):
+                grad_input[0].data *= _gamma
+
+            handles.append(module.register_full_backward_hook(hook))
+        return handles
+
+    def _remove_hooks(self) -> None:
+        for h in self._lsgm_handles:
+            h.remove()
+        self._lsgm_handles.clear()
+
+    # --- Setup ---
+
+    def setup(self, prompt: str, target: str) -> None:
+        self._prepare_prompt(prompt, target)
+        K = self.num_starts
+
+        ids_list = [self._init_optim_ids() for _ in range(K)]
+        self.current_ids = torch.stack(ids_list, dim=0)
+
+        init_losses = self.compute_discrete_loss_batch(self.current_ids)
+        self.flop_counter.count_forward(self.total_seq_len, batch_size=K)
+        self.best_losses = init_losses.tolist()
+        self.best_ids = self.current_ids.clone()
+        self._restart_patience = [0] * K
+        self._momentum_buffer = [None] * K
+
+        self._lsgm_handles = self._register_lsgm_hooks()
+        logger.info(
+            "Claude v2: K=%d restarts, search_width %d→%d, LSGM(%d hooks, gamma=%.2f), momentum=%.2f",
+            K,
+            self.search_width_min,
+            self.search_width_max,
+            len(self._lsgm_handles),
+            self.lsgm_gamma,
+            self.momentum,
+        )
+
+    # --- Step ---
+
+    def step(self, step_num: int) -> tuple[float, float | None, str]:
+        K = self.num_starts
+        n_replace = self._get_n_replace()
+        search_width = self._get_search_width()
+
+        grads = self._compute_batched_gradient(self.best_ids)
+        self.flop_counter.count_forward_backward(self.total_seq_len, batch_size=K)
+
+        with torch.no_grad():
+            for k in range(K):
+                if self._momentum_buffer[k] is None:
+                    self._momentum_buffer[k] = grads[k].clone()
+                else:
+                    self._momentum_buffer[k] = self.momentum * self._momentum_buffer[k] + (1 - self.momentum) * grads[k]
+
+            all_candidates = []
+            restart_sizes = []
+            for k in range(K):
+                sampled = sample_ids_from_grad(
+                    self.best_ids[k],
+                    self._momentum_buffer[k],
+                    search_width,
+                    self.topk_per_position,
+                    n_replace,
+                    not_allowed_ids=self.not_allowed_ids,
+                )
+                all_candidates.append(sampled)
+                restart_sizes.append(sampled.shape[0])
+
+            all_candidates = torch.cat(all_candidates, dim=0)
+            total_candidates = sum(restart_sizes)
+
+            batch_losses = self.compute_discrete_loss_batch(all_candidates)
+            self.flop_counter.count_forward(self.total_seq_len, batch_size=total_candidates)
+
+            offset = 0
+            for k in range(K):
+                sz = restart_sizes[k]
+                restart_losses = batch_losses[offset : offset + sz]
+                best_idx = restart_losses.argmin().item()
+                candidate_loss = restart_losses[best_idx].item()
+
+                self.current_ids[k] = all_candidates[offset + best_idx]
+
+                if candidate_loss < self.best_losses[k]:
+                    self.best_losses[k] = candidate_loss
+                    self.best_ids[k] = self.current_ids[k].clone()
+                    self._restart_patience[k] = 0
+                else:
+                    self._restart_patience[k] += 1
+                offset += sz
+
+            for k in range(K):
+                if self._restart_patience[k] >= self.patience_limit:
+                    self._perturb_restart(k)
+                    self._restart_patience[k] = 0
+
+        self.log("n_replace", n_replace, prog_bar=True)
+        self.log("search_width", search_width)
+
+        best_k = min(range(K), key=lambda k: self.best_losses[k])
+        optim_str = self.tokenizer.decode(self.best_ids[best_k])
+        self._step_ids = self.best_ids[best_k]
+        return self.best_losses[best_k], None, optim_str
+
+    def _perturb_restart(self, k: int) -> None:
+        self.current_ids[k] = self.best_ids[k].clone()
+        positions = torch.randperm(self.optim_length, device=self.current_ids.device)[: self.n_perturb]
+        random_tokens = self.allowed_token_ids[
+            torch.randint(len(self.allowed_token_ids), (self.n_perturb,), device=self.current_ids.device)
+        ]
+        self.current_ids[k, positions] = random_tokens
+        self.best_ids[k] = self.current_ids[k].clone()
+        new_loss = self.compute_discrete_loss(self.current_ids[k])
+        self.flop_counter.count_forward(self.total_seq_len)
+        self.best_losses[k] = new_loss
+        self._momentum_buffer[k] = None
+
+    def _compute_batched_gradient(self, optim_ids: Tensor) -> Tensor:
+        K = optim_ids.shape[0]
+        embedding_layer = self.embedding_layer
+
+        optim_ids_onehot = torch.nn.functional.one_hot(
+            optim_ids,
+            num_classes=embedding_layer.num_embeddings,
+        ).to(self.model.device, self.model.dtype)
+        optim_ids_onehot.requires_grad_(True)
+
+        optim_embeds = optim_ids_onehot @ embedding_layer.weight
+
+        input_embeds = torch.cat(
+            [
+                self.before_embeds.expand(K, -1, -1),
+                optim_embeds,
+                self.after_embeds.expand(K, -1, -1),
+                self.target_embeds.expand(K, -1, -1),
+            ],
+            dim=1,
+        )
+
+        output = self.model(inputs_embeds=input_embeds)
+        logits = output.logits
+
+        shift = input_embeds.shape[1] - self.target_ids.shape[1]
+        target_len = self.target_ids.shape[1]
+        shift_logits = logits[..., shift - 1 : shift - 1 + target_len, :].contiguous()
+
+        target_expanded = self.target_ids.expand(K, -1)
+        losses = torch.nn.functional.cross_entropy(
+            shift_logits.view(-1, shift_logits.size(-1)),
+            target_expanded.reshape(-1),
+            reduction="none",
+        )
+        total_loss = losses.sum()
+
+        grad = torch.autograd.grad(outputs=[total_loss], inputs=[optim_ids_onehot])[0]
+        return grad
+
+    def run(self, prompt, target, num_steps, max_flops=None, max_time=None, **kwargs):
+        self.max_flops = max_flops
+        try:
+            return super().run(prompt, target, num_steps, max_flops=max_flops, max_time=max_time, **kwargs)
+        finally:
+            self._remove_hooks()

claudini/methods/claude_random/v20/__init__.py

@@ -0,0 +1,3 @@
+from .optimizer import ClaudeV20Optimizer
+
+__all__ = ["ClaudeV20Optimizer"]

claudini/methods/claude_random/v20/diagnostics.jsonl

@@ -0,0 +1,490 @@
+{"step": 0, "discrete_loss": 12.766921997070312, "best_sample_loss": 12.679755210876465, "soft_loss": 13.158564567565918, "best_discrete": 12.679755210876465, "best_soft": 13.158564567565918, "best_argmax": 12.766921997070312, "best_sampling": 12.679755210876465, "relax_gap": -0.030676350226427136, "n_match": 19, "g_first_norm": 171.2406463623047, "vocab_size": 50257, "entropy": 1.1097787618637085, "entropy_per_token": [0.7068907022476196, 0.7737476229667664, 1.0852322578430176, 1.944070816040039, 0.8106542825698853, 1.1774251461029053, 1.1739075183868408, 0.6952491402626038, 0.20709764957427979, 0.04646778479218483, 0.2571680545806885, 1.7969164848327637, 0.7661004066467285, 0.9168926477432251, 1.364220380783081, 1.5056180953979492, 1.3868541717529297, 2.321444034576416, 1.7614589929580688, 1.4981589317321777], "max_p": 0.6960645914077759, "max_p_per_token": [0.7763784527778625, 0.8428146839141846, 0.7603229880332947, 0.3690735101699829, 0.7234988212585449, 0.6913996934890747, 0.7771551609039307, 0.8020567893981934, 0.9571436643600464, 0.9942827224731445, 0.9482336640357971, 0.4330495595932007, 0.8235657811164856, 0.7461090683937073, 0.6116974949836731, 0.5747320652008057, 0.5799967646598816, 0.37417036294937134, 0.47310569882392883, 0.6625047922134399], "n_positions_probed": 1, "per_restart_best": [12.679755210876465]}
+{"step": 1, "discrete_loss": 12.766921997070312, "best_sample_loss": 12.639129638671875, "soft_loss": 12.993865966796875, "best_discrete": 12.639129638671875, "best_soft": 12.993865966796875, "best_argmax": 12.766921997070312, "best_sampling": 12.639129638671875, "relax_gap": -0.017775934542299265, "n_match": 18, "g_first_norm": 178.03370666503906, "vocab_size": 50257, "entropy": 1.0815913677215576, "entropy_per_token": [0.685947597026825, 0.8193556070327759, 1.0129082202911377, 1.9828940629959106, 0.7735882997512817, 1.1341925859451294, 1.223487138748169, 0.5084280967712402, 0.1864548623561859, 0.050719283521175385, 0.2911805510520935, 1.7383489608764648, 0.7661162614822388, 0.937384843826294, 1.4510098695755005, 1.4598760604858398, 1.3388234376907349, 2.261017322540283, 1.7337223291397095, 1.276370644569397], "max_p": 0.7059011459350586, "max_p_per_token": [0.7860886454582214, 0.832657516002655, 0.782758891582489, 0.33428099751472473, 0.743806779384613, 0.706419050693512, 0.7641651034355164, 0.872196614742279, 0.9625810384750366, 0.9936574101448059, 0.9389313459396362, 0.4534316956996918, 0.8234569430351257, 0.7385568022727966, 0.5746434926986694, 0.5948060750961304, 0.5911389589309692, 0.3981171250343323, 0.498500257730484, 0.7278288006782532], "n_positions_probed": 1, "per_restart_best": [12.639129638671875]}
+{"step": 2, "discrete_loss": 12.766921997070312, "best_sample_loss": 12.492476463317871, "soft_loss": 12.893331527709961, "best_discrete": 12.492476463317871, "best_soft": 12.893331527709961, "best_argmax": 12.766921997070312, "best_sampling": 12.492476463317871, "relax_gap": -0.009901331790752402, "n_match": 17, "g_first_norm": 144.1359405517578, "vocab_size": 50257, "entropy": 1.089734435081482, "entropy_per_token": [0.6730030179023743, 0.8481442928314209, 1.3664817810058594, 1.9936128854751587, 0.7352871894836426, 1.0842653512954712, 1.265512466430664, 0.47372984886169434, 0.16601300239562988, 0.05498660355806351, 0.3250654637813568, 1.6891452074050903, 0.768416702747345, 0.9524056315422058, 1.4307889938354492, 1.463634967803955, 1.3097257614135742, 2.2043161392211914, 1.6678235530853271, 1.3223292827606201], "max_p": 0.7044004201889038, "max_p_per_token": [0.7910276651382446, 0.8253538608551025, 0.6822192668914795, 0.31060484051704407, 0.7629421353340149, 0.7226108908653259, 0.7522646188735962, 0.884297788143158, 0.9677616357803345, 0.9930176734924316, 0.9291387796401978, 0.4697706997394562, 0.822963535785675, 0.733390212059021, 0.5846369862556458, 0.5951162576675415, 0.5934475660324097, 0.41996780037879944, 0.532019853591919, 0.7154566049575806], "n_positions_probed": 1, "per_restart_best": [12.492476463317871]}
+{"step": 3, "discrete_loss": 12.766921997070312, "best_sample_loss": 12.472553253173828, "soft_loss": 12.862942695617676, "best_discrete": 12.472553253173828, "best_soft": 12.862942695617676, "best_argmax": 12.766921997070312, "best_sampling": 12.472553253173828, "relax_gap": -0.007521053122232408, "n_match": 16, "g_first_norm": 147.49737548828125, "vocab_size": 50257, "entropy": 1.0688852071762085, "entropy_per_token": [0.6531802415847778, 0.8842275142669678, 1.2147570848464966, 2.0295212268829346, 0.6511818766593933, 1.0659143924713135, 1.3085476160049438, 0.4368155598640442, 0.14692460000514984, 0.05985066294670105, 0.36896443367004395, 1.6575185060501099, 0.7725505232810974, 0.9164679050445557, 1.430690884590149, 1.441624641418457, 1.2528122663497925, 2.1452512741088867, 1.655975580215454, 1.2849260568618774], "max_p": 0.7119964957237244, "max_p_per_token": [0.8002871870994568, 0.815993070602417, 0.7306378483772278, 0.29177045822143555, 0.8043175339698792, 0.728569746017456, 0.7392308115959167, 0.8964648246765137, 0.9723964929580688, 0.9922763705253601, 0.9158743023872375, 0.476537823677063, 0.8217534422874451, 0.7470932602882385, 0.5846796631813049, 0.6062793135643005, 0.6089620590209961, 0.4411388039588928, 0.5383795499801636, 0.727286696434021], "n_positions_probed": 1, "per_restart_best": [12.472553253173828]}
+{"step": 4, "discrete_loss": 12.721638679504395, "best_sample_loss": 12.369236946105957, "soft_loss": 12.800943374633789, "best_discrete": 12.369236946105957, "best_soft": 12.800943374633789, "best_argmax": 12.721638679504395, "best_sampling": 12.369236946105957, "relax_gap": -0.006233842756213546, "n_match": 15, "g_first_norm": 143.0030059814453, "vocab_size": 50257, "entropy": 1.059029221534729, "entropy_per_token": [0.6518300771713257, 0.891356348991394, 1.1484405994415283, 2.0137972831726074, 0.7034109830856323, 1.0297985076904297, 1.3611626625061035, 0.4105866253376007, 0.13288554549217224, 0.06562935560941696, 0.40479081869125366, 1.6357228755950928, 0.7786425352096558, 0.9410779476165771, 1.3844369649887085, 1.4595463275909424, 1.1752992868423462, 2.103525161743164, 1.6249306201934814, 1.2637128829956055], "max_p": 0.7141801714897156, "max_p_per_token": [0.7981722950935364, 0.8143057227134705, 0.7505185604095459, 0.27865758538246155, 0.7840676307678223, 0.7392271757125854, 0.723155677318573, 0.904870867729187, 0.9756807088851929, 0.9913806319236755, 0.9046424031257629, 0.4804892838001251, 0.8200535178184509, 0.7389175295829773, 0.6052815318107605, 0.6021786332130432, 0.6325035691261292, 0.4548245370388031, 0.5503767728805542, 0.7342979311943054], "n_positions_probed": 1, "per_restart_best": [12.369236946105957]}
+{"step": 5, "discrete_loss": 12.721638679504395, "best_sample_loss": 12.280828475952148, "soft_loss": 12.724222183227539, "best_discrete": 12.280828475952148, "best_soft": 12.724222183227539, "best_argmax": 12.721638679504395, "best_sampling": 12.280828475952148, "relax_gap": -0.00020307947649124544, "n_match": 14, "g_first_norm": 194.90025329589844, "vocab_size": 50257, "entropy": 1.0258591175079346, "entropy_per_token": [0.6443796157836914, 0.8963103294372559, 1.0829261541366577, 2.0004494190216064, 0.701981782913208, 0.873299777507782, 1.4142040014266968, 0.387592613697052, 0.12108801305294037, 0.07219330221414566, 0.439880907535553, 1.6277567148208618, 0.7848072052001953, 0.9432525634765625, 1.3390886783599854, 1.451371669769287, 0.8175164461135864, 2.0761680603027344, 1.5939698219299316, 1.248944878578186], "max_p": 0.7262465357780457, "max_p_per_token": [0.7968822121620178, 0.8131501078605652, 0.7693901658058167, 0.30001720786094666, 0.7850956916809082, 0.7822403907775879, 0.7060919404029846, 0.9120599031448364, 0.9783597588539124, 0.9903433322906494, 0.893458366394043, 0.47886982560157776, 0.8182309865951538, 0.7390251755714417, 0.6240571141242981, 0.6096639037132263, 0.7662956714630127, 0.4622182250022888, 0.5601296424865723, 0.7393518686294556], "n_positions_probed": 1, "per_restart_best": [12.280828475952148]}
+{"step": 6, "discrete_loss": 12.721638679504395, "best_sample_loss": 12.314135551452637, "soft_loss": 12.596317291259766, "best_discrete": 12.280828475952148, "best_soft": 12.596317291259766, "best_argmax": 12.721638679504395, "best_sampling": 12.280828475952148, "relax_gap": 0.00985104131644078, "n_match": 14, "g_first_norm": 122.53971099853516, "vocab_size": 50257, "entropy": 1.0293911695480347, "entropy_per_token": [0.6447609663009644, 0.8822052478790283, 1.0314500331878662, 1.988631248474121, 0.6944905519485474, 0.8940011858940125, 1.6436724662780762, 0.36766552925109863, 0.1116732731461525, 0.07823432981967926, 0.46769046783447266, 1.5993764400482178, 0.7921229600906372, 0.9309527277946472, 1.3782033920288086, 1.3986440896987915, 0.8481186628341675, 2.0857656002044678, 1.5609710216522217, 1.189192771911621], "max_p": 0.7255215048789978, "max_p_per_token": [0.7973021268844604, 0.8170415163040161, 0.7836745381355286, 0.3229716420173645, 0.7889315485954285, 0.7747706174850464, 0.6355904936790466, 0.9181160926818848, 0.980440080165863, 0.9893653392791748, 0.8842958211898804, 0.4890024960041046, 0.8157781958580017, 0.7436088919639587, 0.6052818894386292, 0.6334131360054016, 0.7555397152900696, 0.45039504766464233, 0.5685209631919861, 0.7563902735710144], "n_positions_probed": 1, "per_restart_best": [12.280828475952148]}
+{"step": 7, "discrete_loss": 12.721638679504395, "best_sample_loss": 12.256601333618164, "soft_loss": 12.549005508422852, "best_discrete": 12.256601333618164, "best_soft": 12.549005508422852, "best_argmax": 12.721638679504395, "best_sampling": 12.256601333618164, "relax_gap": 0.01357004199149825, "n_match": 13, "g_first_norm": 123.31951904296875, "vocab_size": 50257, "entropy": 1.0177639722824097, "entropy_per_token": [0.6450796723365784, 0.8670791387557983, 0.9881473779678345, 1.9714525938034058, 0.7107703685760498, 0.9053350687026978, 1.67705237865448, 0.34291088581085205, 0.1030501127243042, 0.08471380174160004, 0.49313876032829285, 1.5784600973129272, 0.803528904914856, 0.924974799156189, 1.2640364170074463, 1.3567678928375244, 0.8762257695198059, 2.08294415473938, 1.5160553455352783, 1.1635565757751465], "max_p": 0.7295661568641663, "max_p_per_token": [0.7963941097259521, 0.8211801052093506, 0.7953912615776062, 0.34474316239356995, 0.7811076045036316, 0.77020263671875, 0.6228182315826416, 0.9243913888931274, 0.9822991490364075, 0.9882986545562744, 0.8758798837661743, 0.49649590253829956, 0.8121391534805298, 0.7460629940032959, 0.6509850025177002, 0.6505116820335388, 0.7453445196151733, 0.4426378011703491, 0.5804993510246277, 0.7639396786689758], "n_positions_probed": 1, "per_restart_best": [12.256601333618164]}
+{"step": 8, "discrete_loss": 12.721638679504395, "best_sample_loss": 12.22569751739502, "soft_loss": 12.54110336303711, "best_discrete": 12.22569751739502, "best_soft": 12.54110336303711, "best_argmax": 12.721638679504395, "best_sampling": 12.22569751739502, "relax_gap": 0.014191199814387307, "n_match": 12, "g_first_norm": 124.26081848144531, "vocab_size": 50257, "entropy": 1.0382722616195679, "entropy_per_token": [0.65889972448349, 0.8580150008201599, 0.9521125555038452, 1.9610731601715088, 0.6979612708091736, 0.9080787897109985, 1.701221227645874, 0.32692331075668335, 0.22156599164009094, 0.09147673100233078, 0.5105787515640259, 1.5527366399765015, 0.813502311706543, 0.9249098896980286, 1.4795677661895752, 1.5317480564117432, 0.9024065732955933, 2.0714993476867676, 1.4673664569854736, 1.1338012218475342], "max_p": 0.721466600894928, "max_p_per_token": [0.790277361869812, 0.8235452175140381, 0.8049511313438416, 0.3649826943874359, 0.7876364588737488, 0.7689746022224426, 0.6119741201400757, 0.92913818359375, 0.9498534798622131, 0.9871624112129211, 0.8702419996261597, 0.5049669742584229, 0.809029757976532, 0.7464528679847717, 0.5545367002487183, 0.5874406695365906, 0.7358178496360779, 0.4379284977912903, 0.591945469379425, 0.7724761366844177], "n_positions_probed": 1, "per_restart_best": [12.22569751739502]}
+{"step": 9, "discrete_loss": 12.721638679504395, "best_sample_loss": 12.11977767944336, "soft_loss": 12.518335342407227, "best_discrete": 12.11977767944336, "best_soft": 12.518335342407227, "best_argmax": 12.721638679504395, "best_sampling": 12.11977767944336, "relax_gap": 0.015980907980408715, "n_match": 11, "g_first_norm": 122.49620819091797, "vocab_size": 50257, "entropy": 1.0342124700546265, "entropy_per_token": [0.6593402624130249, 0.8491158485412598, 0.9176709651947021, 1.9281394481658936, 0.7312172651290894, 0.9091067314147949, 1.7345759868621826, 0.3113660216331482, 0.20098645985126495, 0.15233442187309265, 0.5366443395614624, 1.556532621383667, 0.8311086297035217, 0.9248573780059814, 1.4681298732757568, 1.4368025064468384, 0.9199636578559875, 2.0767955780029297, 1.4135875701904297, 1.1259726285934448], "max_p": 0.7230066657066345, "max_p_per_token": [0.7897917032241821, 0.8259698152542114, 0.8139348030090332, 0.3929142951965332, 0.7719266414642334, 0.7679082751274109, 0.5979592800140381, 0.933735728263855, 0.9560129046440125, 0.9762988686561584, 0.8616201281547546, 0.5012252926826477, 0.8034834265708923, 0.746728777885437, 0.5581910014152527, 0.6268957257270813, 0.7303599119186401, 0.42444461584091187, 0.6056562662124634, 0.7750750780105591], "n_positions_probed": 1, "per_restart_best": [12.11977767944336]}
+{"step": 10, "discrete_loss": 12.721638679504395, "best_sample_loss": 12.025465965270996, "soft_loss": 12.465242385864258, "best_discrete": 12.025465965270996, "best_soft": 12.465242385864258, "best_argmax": 12.721638679504395, "best_sampling": 12.025465965270996, "relax_gap": 0.020154344900017655, "n_match": 10, "g_first_norm": 121.22233581542969, "vocab_size": 50257, "entropy": 1.0215225219726562, "entropy_per_token": [0.6574990153312683, 0.8445888757705688, 0.8868539333343506, 1.9028080701828003, 0.7228846549987793, 0.9116863012313843, 1.7597322463989258, 0.29600411653518677, 0.1841161847114563, 0.16312381625175476, 0.48355832695961, 1.554124355316162, 0.8486148118972778, 0.913909912109375, 1.4682307243347168, 1.3425254821777344, 0.9311200380325317, 2.0751047134399414, 1.360966682434082, 1.1229968070983887], "max_p": 0.7284662127494812, "max_p_per_token": [0.7909766435623169, 0.8272007703781128, 0.8218024969100952, 0.41642868518829346, 0.7767425179481506, 0.766434907913208, 0.585950493812561, 0.9381637573242188, 0.9608602523803711, 0.9742643237113953, 0.9118886590003967, 0.5000421404838562, 0.7979434728622437, 0.750900149345398, 0.5533314347267151, 0.6619961261749268, 0.7270532250404358, 0.4137735664844513, 0.6172291040420532, 0.7763421535491943], "n_positions_probed": 1, "per_restart_best": [12.025465965270996]}
+{"step": 11, "discrete_loss": 12.721638679504395, "best_sample_loss": 11.975626945495605, "soft_loss": 12.419515609741211, "best_discrete": 11.975626945495605, "best_soft": 12.419515609741211, "best_argmax": 12.721638679504395, "best_sampling": 11.975626945495605, "relax_gap": 0.0237487541797527, "n_match": 9, "g_first_norm": 121.02429962158203, "vocab_size": 50257, "entropy": 1.026016116142273, "entropy_per_token": [0.6603832244873047, 0.8422136306762695, 0.8631768226623535, 1.8880062103271484, 0.7163380980491638, 0.9194862842559814, 1.7805509567260742, 0.28048276901245117, 0.17119862139225006, 0.1740826666355133, 0.533211350440979, 1.6872659921646118, 0.8660762310028076, 0.9032765030860901, 1.4702467918395996, 1.3608304262161255, 0.9363787770271301, 2.0683040618896484, 1.2906200885772705, 1.1081922054290771], "max_p": 0.725806713104248, "max_p_per_token": [0.7894768118858337, 0.8278316259384155, 0.8277244567871094, 0.43384504318237305, 0.7807818651199341, 0.7630049586296082, 0.5748458504676819, 0.942520022392273, 0.9644566774368286, 0.9721664190292358, 0.8990588188171387, 0.4503324329853058, 0.7924432754516602, 0.7547227740287781, 0.54250168800354, 0.6558084487915039, 0.7255290746688843, 0.4050108790397644, 0.6334350109100342, 0.780638575553894], "n_positions_probed": 1, "per_restart_best": [11.975626945495605]}
+{"step": 12, "discrete_loss": 12.593037605285645, "best_sample_loss": 11.784964561462402, "soft_loss": 12.380243301391602, "best_discrete": 11.784964561462402, "best_soft": 12.380243301391602, "best_argmax": 12.593037605285645, "best_sampling": 11.784964561462402, "relax_gap": 0.016897774037038318, "n_match": 8, "g_first_norm": 128.0647430419922, "vocab_size": 50257, "entropy": 1.076188564300537, "entropy_per_token": [0.6635771989822388, 0.838215172290802, 0.842634916305542, 1.8842597007751465, 0.7010859251022339, 0.9262485504150391, 1.7960357666015625, 0.26578670740127563, 0.16025222837924957, 0.18572643399238586, 0.5845844745635986, 1.6628508567810059, 1.9919469356536865, 0.8991720080375671, 1.4893794059753418, 1.3496198654174805, 0.9390636682510376, 2.062224864959717, 1.1919344663619995, 1.0891731977462769], "max_p": 0.7035762667655945, "max_p_per_token": [0.788021981716156, 0.8288134932518005, 0.8327566981315613, 0.44394198060035706, 0.7888922095298767, 0.7596255540847778, 0.5650720000267029, 0.946495532989502, 0.967409610748291, 0.9699088931083679, 0.8849672675132751, 0.46228593587875366, 0.33869925141334534, 0.7560540437698364, 0.5130124092102051, 0.6598913073539734, 0.7251030802726746, 0.3970308005809784, 0.6576661467552185, 0.7858776450157166], "n_positions_probed": 1, "per_restart_best": [11.784964561462402]}
+{"step": 13, "discrete_loss": 12.325775146484375, "best_sample_loss": 11.543366432189941, "soft_loss": 12.20728874206543, "best_discrete": 11.543366432189941, "best_soft": 12.20728874206543, "best_argmax": 12.325775146484375, "best_sampling": 11.543366432189941, "relax_gap": 0.009612896796412893, "n_match": 8, "g_first_norm": 203.70973205566406, "vocab_size": 50257, "entropy": 1.0366697311401367, "entropy_per_token": [0.6735405325889587, 0.817916750907898, 0.8295230865478516, 1.899888515472412, 0.7072038054466248, 0.9323489665985107, 1.8204104900360107, 0.24984857439994812, 0.15255475044250488, 0.1994137018918991, 0.6466037034988403, 1.6785664558410645, 1.967128038406372, 0.09422153979539871, 1.5190670490264893, 1.4286394119262695, 0.9498695731163025, 2.0718770027160645, 1.0654096603393555, 1.0293641090393066], "max_p": 0.7070862650871277, "max_p_per_token": [0.7823284268379211, 0.8339908123016357, 0.835841178894043, 0.44250160455703735, 0.7856238484382629, 0.7559583783149719, 0.5530272126197815, 0.9504290223121643, 0.9694560170173645, 0.9672194123268127, 0.8673185706138611, 0.4517667293548584, 0.35658228397369385, 0.9831695556640625, 0.3909744620323181, 0.6314370036125183, 0.7217987775802612, 0.375508576631546, 0.6857206225395203, 0.8010733127593994], "n_positions_probed": 1, "per_restart_best": [11.543366432189941]}
+{"step": 14, "discrete_loss": 12.325775146484375, "best_sample_loss": 11.739116668701172, "soft_loss": 12.210139274597168, "best_discrete": 11.543366432189941, "best_soft": 12.20728874206543, "best_argmax": 12.325775146484375, "best_sampling": 11.543366432189941, "relax_gap": 0.009381630811283242, "n_match": 8, "g_first_norm": 134.14984130859375, "vocab_size": 50257, "entropy": 1.029180884361267, "entropy_per_token": [0.6815602779388428, 0.8237155675888062, 0.8087214231491089, 1.8171684741973877, 0.7134277820587158, 0.9259452223777771, 1.7779033184051514, 0.24647927284240723, 0.13836722075939178, 0.20852209627628326, 0.6884951591491699, 1.648929238319397, 1.8808985948562622, 0.09674539417028427, 1.6897101402282715, 1.3747925758361816, 0.9413720369338989, 2.074859619140625, 1.0268924236297607, 1.0191117525100708], "max_p": 0.711514413356781, "max_p_per_token": [0.7778928875923157, 0.8326735496520996, 0.8409550189971924, 0.4848870038986206, 0.7826067209243774, 0.7563562989234924, 0.5639777779579163, 0.9513707756996155, 0.9731140732765198, 0.9653520584106445, 0.8545794486999512, 0.48002684116363525, 0.4184577763080597, 0.9826199412345886, 0.3300721347332001, 0.650576651096344, 0.7265123128890991, 0.36392009258270264, 0.6907520890235901, 0.803584635257721], "n_positions_probed": 1, "per_restart_best": [11.543366432189941]}
+{"step": 15, "discrete_loss": 12.05322551727295, "best_sample_loss": 11.195345878601074, "soft_loss": 12.129611015319824, "best_discrete": 11.195345878601074, "best_soft": 12.129611015319824, "best_argmax": 12.05322551727295, "best_sampling": 11.195345878601074, "relax_gap": -0.006337349113514079, "n_match": 7, "g_first_norm": 132.68838500976562, "vocab_size": 50257, "entropy": 0.9601629376411438, "entropy_per_token": [0.6845604181289673, 0.8380334377288818, 0.7994986176490784, 1.7843340635299683, 0.6927849054336548, 0.9326156377792358, 1.7596194744110107, 0.24192950129508972, 0.12868157029151917, 0.2171933650970459, 0.713088870048523, 1.606650471687317, 1.7404650449752808, 0.09961672127246857, 1.658471703529358, 0.22991755604743958, 0.9323670864105225, 2.081702709197998, 1.0666449069976807, 0.9950825572013855], "max_p": 0.7305841445922852, "max_p_per_token": [0.777087926864624, 0.8291372656822205, 0.8430564999580383, 0.5018265247344971, 0.79306560754776, 0.7519525289535522, 0.5668705105781555, 0.9526234865188599, 0.975532054901123, 0.9635700583457947, 0.8463562726974487, 0.5100246071815491, 0.4989992082118988, 0.9819728136062622, 0.3063831031322479, 0.9535055160522461, 0.7309202551841736, 0.35150346159935, 0.6678575277328491, 0.8094367384910583], "n_positions_probed": 1, "per_restart_best": [11.195345878601074]}
+{"step": 16, "discrete_loss": 12.05322551727295, "best_sample_loss": 11.098593711853027, "soft_loss": 11.912351608276367, "best_discrete": 11.098593711853027, "best_soft": 11.912351608276367, "best_argmax": 12.05322551727295, "best_sampling": 11.098593711853027, "relax_gap": 0.011687652304746294, "n_match": 6, "g_first_norm": 125.20123291015625, "vocab_size": 50257, "entropy": 0.9644104242324829, "entropy_per_token": [0.6765528321266174, 0.8728533387184143, 0.8031398057937622, 1.8152281045913696, 0.7097638249397278, 0.9469990730285645, 1.7558624744415283, 0.23423205316066742, 0.11768986284732819, 0.22465376555919647, 0.7201566100120544, 1.6149590015411377, 1.5466313362121582, 0.1029050350189209, 1.625575304031372, 0.2221015840768814, 0.9216803908348083, 2.077965021133423, 1.102384328842163, 1.1968741416931152], "max_p": 0.7295047044754028, "max_p_per_token": [0.7825636267662048, 0.8203525543212891, 0.8418926000595093, 0.4884886145591736, 0.7857025265693665, 0.7445160150527954, 0.5648062229156494, 0.9546616673469543, 0.9781936407089233, 0.9619724750518799, 0.8435963988304138, 0.510322630405426, 0.5890750288963318, 0.9812265634536743, 0.2960509657859802, 0.955726683139801, 0.736621081829071, 0.34551066160202026, 0.6513396501541138, 0.7574740052223206], "n_positions_probed": 1, "per_restart_best": [11.098593711853027]}
+{"step": 17, "discrete_loss": 12.05322551727295, "best_sample_loss": 11.017462730407715, "soft_loss": 11.780887603759766, "best_discrete": 11.017462730407715, "best_soft": 11.780887603759766, "best_argmax": 12.05322551727295, "best_sampling": 11.017462730407715, "relax_gap": 0.022594608648357908, "n_match": 5, "g_first_norm": 127.18212890625, "vocab_size": 50257, "entropy": 0.9622918367385864, "entropy_per_token": [0.6632212996482849, 0.9097033739089966, 0.8072787523269653, 1.8390616178512573, 0.7099494934082031, 0.96700119972229, 1.736991047859192, 0.2267698347568512, 0.1101403534412384, 0.23290428519248962, 0.7226672172546387, 1.6182279586791992, 1.3876210451126099, 0.10838115215301514, 1.5958433151245117, 0.2147998809814453, 0.9172861576080322, 2.026066303253174, 1.2030465602874756, 1.2488754987716675], "max_p": 0.7258356809616089, "max_p_per_token": [0.7909252047538757, 0.8109164834022522, 0.8405811190605164, 0.47961387038230896, 0.786329448223114, 0.7352067232131958, 0.5683158040046692, 0.9565994143486023, 0.9799802303314209, 0.9602147340774536, 0.8421052694320679, 0.5105913281440735, 0.6525658965110779, 0.9799538850784302, 0.29510876536369324, 0.9577736854553223, 0.7384729981422424, 0.2733812630176544, 0.6147723197937012, 0.7433049082756042], "n_positions_probed": 1, "per_restart_best": [11.017462730407715]}
+{"step": 18, "discrete_loss": 12.490243911743164, "best_sample_loss": 11.098945617675781, "soft_loss": 11.722192764282227, "best_discrete": 11.017462730407715, "best_soft": 11.722192764282227, "best_argmax": 12.05322551727295, "best_sampling": 11.017462730407715, "relax_gap": 0.061492085573991544, "n_match": 4, "g_first_norm": 131.26858520507812, "vocab_size": 50257, "entropy": 0.978663444519043, "entropy_per_token": [0.6601279973983765, 0.9332653284072876, 0.8131063580513, 1.8569178581237793, 0.7176970839500427, 0.980243444442749, 1.712378978729248, 0.21983025968074799, 0.10335344821214676, 0.236352801322937, 0.7319580316543579, 1.6127382516860962, 1.5445733070373535, 0.11371462047100067, 1.5652711391448975, 0.2070927917957306, 0.904107928276062, 1.9920735359191895, 1.409029245376587, 1.2594351768493652], "max_p": 0.7200390696525574, "max_p_per_token": [0.7940142154693604, 0.804763913154602, 0.8388532400131226, 0.47310495376586914, 0.7834476232528687, 0.7282686233520508, 0.5747721195220947, 0.9583219289779663, 0.9815471172332764, 0.9594614505767822, 0.8385273814201355, 0.5156717300415039, 0.5917812585830688, 0.9786524772644043, 0.2922361493110657, 0.9598349332809448, 0.7441088557243347, 0.27002841234207153, 0.5730825662612915, 0.7403019070625305], "n_positions_probed": 1, "per_restart_best": [11.017462730407715]}
+{"step": 19, "discrete_loss": 12.46641731262207, "best_sample_loss": 11.074057579040527, "soft_loss": 11.684500694274902, "best_discrete": 11.017462730407715, "best_soft": 11.684500694274902, "best_argmax": 12.05322551727295, "best_sampling": 11.017462730407715, "relax_gap": 0.06272183890037826, "n_match": 4, "g_first_norm": 133.87130737304688, "vocab_size": 50257, "entropy": 0.9779146313667297, "entropy_per_token": [0.6390300989151001, 0.9664912223815918, 0.8115776777267456, 1.8311700820922852, 0.6883102059364319, 0.9832710027694702, 1.6889870166778564, 0.21308542788028717, 0.09512725472450256, 0.24404200911521912, 0.752858579158783, 1.6425557136535645, 1.4025685787200928, 0.1201113685965538, 1.5343958139419556, 0.20032745599746704, 0.8864847421646118, 1.9539003372192383, 1.4332106113433838, 1.470787763595581], "max_p": 0.7209199070930481, "max_p_per_token": [0.805486798286438, 0.79603111743927, 0.8389686346054077, 0.48648959398269653, 0.7973892092704773, 0.7250193357467651, 0.5801572203636169, 0.959984302520752, 0.9833962321281433, 0.957798957824707, 0.8313055634498596, 0.5000267028808594, 0.6484171152114868, 0.9770943522453308, 0.3175968527793884, 0.9616649150848389, 0.7516036033630371, 0.27818578481674194, 0.5620276927947998, 0.6597545146942139], "n_positions_probed": 1, "per_restart_best": [11.017462730407715]}
+{"step": 20, "discrete_loss": 12.46641731262207, "best_sample_loss": 11.019126892089844, "soft_loss": 11.6347074508667, "best_discrete": 11.017462730407715, "best_soft": 11.6347074508667, "best_argmax": 12.05322551727295, "best_sampling": 11.017462730407715, "relax_gap": 0.0667160292246335, "n_match": 4, "g_first_norm": 137.7208251953125, "vocab_size": 50257, "entropy": 0.9614161849021912, "entropy_per_token": [0.3036075234413147, 0.986103892326355, 0.8132694959640503, 1.8511220216751099, 0.6389215588569641, 0.9754555225372314, 1.665330410003662, 0.2057788074016571, 0.08886787295341492, 0.24652911722660065, 0.7640120983123779, 1.6451451778411865, 1.5448918342590332, 0.12592166662216187, 1.508587121963501, 0.19394370913505554, 0.8801220655441284, 1.9079954624176025, 1.5060807466506958, 1.3766371011734009], "max_p": 0.7265598177909851, "max_p_per_token": [0.9325735569000244, 0.7908390760421753, 0.838211178779602, 0.4780614376068115, 0.8195063471794128, 0.7256107330322266, 0.5863931179046631, 0.9617400169372559, 0.9847669005393982, 0.9572453498840332, 0.8268358111381531, 0.4983889162540436, 0.5937981009483337, 0.9756174087524414, 0.32638612389564514, 0.9633311629295349, 0.7547748684883118, 0.2913168668746948, 0.5360101461410522, 0.6897888779640198], "n_positions_probed": 1, "per_restart_best": [11.017462730407715]}
+{"step": 21, "discrete_loss": 12.46641731262207, "best_sample_loss": 11.060295104980469, "soft_loss": 11.567853927612305, "best_discrete": 11.017462730407715, "best_soft": 11.567853927612305, "best_argmax": 12.05322551727295, "best_sampling": 11.017462730407715, "relax_gap": 0.07207871856655905, "n_match": 4, "g_first_norm": 137.0992889404297, "vocab_size": 50257, "entropy": 0.9574571847915649, "entropy_per_token": [0.29960525035858154, 0.99802166223526, 0.8146755695343018, 1.8399823904037476, 0.607215166091919, 0.9599408507347107, 1.6555461883544922, 0.20045289397239685, 0.08300014585256577, 0.25279325246810913, 0.7862882614135742, 1.6666738986968994, 1.4516524076461792, 0.13357782363891602, 1.4776008129119873, 0.18793362379074097, 0.8721380233764648, 1.8664309978485107, 1.5743988752365112, 1.421213150024414], "max_p": 0.7280609011650085, "max_p_per_token": [0.9341586232185364, 0.7884527444839478, 0.8375415802001953, 0.4833911061286926, 0.8329086899757385, 0.7291072607040405, 0.5872397422790527, 0.9629894495010376, 0.9860235452651978, 0.9558620452880859, 0.8187834024429321, 0.4858910143375397, 0.6318064332008362, 0.9736645817756653, 0.34442850947380066, 0.9649038910865784, 0.7584697008132935, 0.300618976354599, 0.5078131556510925, 0.6771630048751831], "n_positions_probed": 1, "per_restart_best": [11.017462730407715]}
+{"step": 22, "discrete_loss": 12.46641731262207, "best_sample_loss": 11.024967193603516, "soft_loss": 11.522485733032227, "best_discrete": 11.017462730407715, "best_soft": 11.522485733032227, "best_argmax": 12.05322551727295, "best_sampling": 11.017462730407715, "relax_gap": 0.07571795135031509, "n_match": 4, "g_first_norm": 136.89830017089844, "vocab_size": 50257, "entropy": 0.9418145418167114, "entropy_per_token": [0.3018965721130371, 1.000751256942749, 0.4186326265335083, 1.8657701015472412, 0.592337965965271, 0.9451017379760742, 1.6417875289916992, 0.19433526694774628, 0.0779610425233841, 0.25550156831741333, 0.8003427982330322, 1.6763370037078857, 1.57289719581604, 0.14167281985282898, 1.4513986110687256, 0.18198858201503754, 0.8649213910102844, 1.8281745910644531, 1.6306809186935425, 1.393801212310791], "max_p": 0.7298116683959961, "max_p_per_token": [0.9339524507522583, 0.7876641750335693, 0.9296634197235107, 0.4705412983894348, 0.8389532566070557, 0.7324164509773254, 0.5897606611251831, 0.9644347429275513, 0.9870801568031311, 0.9552400708198547, 0.8132858872413635, 0.47948142886161804, 0.5847898721694946, 0.9715408086776733, 0.3472083508968353, 0.9664238095283508, 0.7618739604949951, 0.3127393126487732, 0.4830702841281891, 0.6861128211021423], "n_positions_probed": 1, "per_restart_best": [11.017462730407715]}
+{"step": 23, "discrete_loss": 12.46641731262207, "best_sample_loss": 11.06008243560791, "soft_loss": 11.493512153625488, "best_discrete": 11.017462730407715, "best_soft": 11.493512153625488, "best_argmax": 12.05322551727295, "best_sampling": 11.017462730407715, "relax_gap": 0.07804208174641558, "n_match": 4, "g_first_norm": 135.75743103027344, "vocab_size": 50257, "entropy": 0.9394570589065552, "entropy_per_token": [0.29904210567474365, 1.0088645219802856, 0.42944836616516113, 1.8921059370040894, 0.5740249752998352, 0.927710771560669, 1.632810354232788, 0.18954813480377197, 0.0727960467338562, 0.2625117897987366, 0.8175437450408936, 1.6991405487060547, 1.4997578859329224, 0.15069130063056946, 1.415332555770874, 0.17658796906471252, 0.8578956127166748, 1.7903387546539307, 1.641850233078003, 1.4511399269104004], "max_p": 0.7316438555717468, "max_p_per_token": [0.9351429343223572, 0.785361111164093, 0.9273445010185242, 0.47957494854927063, 0.846189022064209, 0.7366887331008911, 0.5903803706169128, 0.965568482875824, 0.9881424307823181, 0.9536774754524231, 0.8068143129348755, 0.4648996889591217, 0.6154049634933472, 0.9691473245620728, 0.37212318181991577, 0.9678093791007996, 0.7652648687362671, 0.32097187638282776, 0.4730527698993683, 0.6693187355995178], "n_positions_probed": 1, "per_restart_best": [11.017462730407715]}
+{"step": 24, "discrete_loss": 12.46641731262207, "best_sample_loss": 11.011818885803223, "soft_loss": 11.456257820129395, "best_discrete": 11.011818885803223, "best_soft": 11.456257820129395, "best_argmax": 12.05322551727295, "best_sampling": 11.011818885803223, "relax_gap": 0.08103045703996317, "n_match": 4, "g_first_norm": 134.94931030273438, "vocab_size": 50257, "entropy": 0.9459850192070007, "entropy_per_token": [0.3004435896873474, 1.0115671157836914, 0.4399866759777069, 1.908646821975708, 0.706468939781189, 0.9128293395042419, 1.6198196411132812, 0.18362560868263245, 0.06829601526260376, 0.2664012014865875, 0.8277941942214966, 1.7106789350509644, 1.5629394054412842, 0.15987014770507812, 1.3899962902069092, 0.1713934689760208, 0.852936863899231, 1.756842851638794, 1.654618501663208, 1.4145451784133911], "max_p": 0.7291234731674194, "max_p_per_token": [0.9351398944854736, 0.7844647765159607, 0.9250412583351135, 0.4708903431892395, 0.8248202800750732, 0.7401441335678101, 0.5926103591918945, 0.9669668078422546, 0.9890487194061279, 0.9527859687805176, 0.8025853633880615, 0.4560745358467102, 0.5914295315742493, 0.9666349291801453, 0.37062644958496094, 0.9691175818443298, 0.7677974700927734, 0.3322155773639679, 0.4631142020225525, 0.6809610724449158], "n_positions_probed": 1, "per_restart_best": [11.011818885803223]}
+{"step": 25, "discrete_loss": 12.46641731262207, "best_sample_loss": 11.010008811950684, "soft_loss": 11.43450927734375, "best_discrete": 11.010008811950684, "best_soft": 11.43450927734375, "best_argmax": 12.05322551727295, "best_sampling": 11.010008811950684, "relax_gap": 0.08277502745183479, "n_match": 4, "g_first_norm": 134.94383239746094, "vocab_size": 50257, "entropy": 0.9479552507400513, "entropy_per_token": [0.29857832193374634, 1.0165255069732666, 0.4503732919692993, 1.8787811994552612, 0.6839907765388489, 1.0570006370544434, 1.6099766492843628, 0.1787382960319519, 0.06396935880184174, 0.2723168134689331, 0.8388648629188538, 1.7226707935333252, 1.5231235027313232, 0.16940689086914062, 1.3561816215515137, 0.16649934649467468, 0.8485183715820312, 1.7237299680709839, 1.6504566669464111, 1.449401617050171], "max_p": 0.7293851971626282, "max_p_per_token": [0.9360197186470032, 0.782895565032959, 0.9227457642555237, 0.48250460624694824, 0.8321104645729065, 0.7066431045532227, 0.59378582239151, 0.9681131839752197, 0.9899030923843384, 0.9514418840408325, 0.7980824708938599, 0.4466680884361267, 0.6082897782325745, 0.9639739990234375, 0.39229458570480347, 0.9703459739685059, 0.770155668258667, 0.342399924993515, 0.4585643708705902, 0.6707663536071777], "n_positions_probed": 1, "per_restart_best": [11.010008811950684]}
+{"step": 26, "discrete_loss": 12.543461799621582, "best_sample_loss": 10.98304557800293, "soft_loss": 11.416769027709961, "best_discrete": 10.98304557800293, "best_soft": 11.416769027709961, "best_argmax": 12.05322551727295, "best_sampling": 10.98304557800293, "relax_gap": 0.08982311182592446, "n_match": 3, "g_first_norm": 134.49917602539062, "vocab_size": 50257, "entropy": 0.9220927357673645, "entropy_per_token": [0.30000099539756775, 1.0188227891921997, 0.46039485931396484, 1.9086782932281494, 0.6698415279388428, 1.0303103923797607, 1.1022987365722656, 0.17322498559951782, 0.059832267463207245, 0.2760215997695923, 0.8468135595321655, 1.7304716110229492, 1.577372670173645, 0.17900700867176056, 1.3341995477676392, 0.1618291139602661, 0.8450671434402466, 1.6938854455947876, 1.652532696723938, 1.421248435974121], "max_p": 0.7333680987358093, "max_p_per_token": [0.9359826445579529, 0.7819496393203735, 0.9204901456832886, 0.46538931131362915, 0.8362392783164978, 0.7143149971961975, 0.7007042169570923, 0.9693880081176758, 0.9907035827636719, 0.9505768418312073, 0.7945671081542969, 0.4394248425960541, 0.5876139998435974, 0.9612137079238892, 0.3878198266029358, 0.9715008735656738, 0.7720941305160522, 0.35498887300491333, 0.45268768072128296, 0.6797125935554504], "n_positions_probed": 1, "per_restart_best": [10.98304557800293]}
+{"step": 27, "discrete_loss": 12.543461799621582, "best_sample_loss": 10.853153228759766, "soft_loss": 11.605310440063477, "best_discrete": 10.853153228759766, "best_soft": 11.416769027709961, "best_argmax": 12.05322551727295, "best_sampling": 10.853153228759766, "relax_gap": 0.07479206095931254, "n_match": 3, "g_first_norm": 150.66488647460938, "vocab_size": 50257, "entropy": 0.9059060215950012, "entropy_per_token": [0.3002638816833496, 1.0409537553787231, 0.469453364610672, 1.8744354248046875, 0.6651696562767029, 1.0231437683105469, 1.0974711179733276, 0.17078687250614166, 0.053989291191101074, 0.2859702408313751, 0.8618109822273254, 1.7270005941390991, 1.4485342502593994, 0.19100309908390045, 1.2780933380126953, 0.15640921890735626, 0.8382473587989807, 1.633905291557312, 1.6451516151428223, 1.3563282489776611], "max_p": 0.7402400970458984, "max_p_per_token": [0.9363458156585693, 0.7757396697998047, 0.9184495210647583, 0.477393239736557, 0.8379321098327637, 0.7128753066062927, 0.6948264241218567, 0.9701003432273865, 0.9918067455291748, 0.9484159350395203, 0.7876729369163513, 0.4381541311740875, 0.6371058225631714, 0.9576767086982727, 0.45324456691741943, 0.9727746248245239, 0.7755704522132874, 0.3778747022151947, 0.4414104223251343, 0.6994326710700989], "n_positions_probed": 1, "per_restart_best": [10.853153228759766]}
+{"step": 28, "discrete_loss": 12.543461799621582, "best_sample_loss": 10.81745433807373, "soft_loss": 11.560391426086426, "best_discrete": 10.81745433807373, "best_soft": 11.416769027709961, "best_argmax": 12.05322551727295, "best_sampling": 10.81745433807373, "relax_gap": 0.07837313089794828, "n_match": 3, "g_first_norm": 147.09588623046875, "vocab_size": 50257, "entropy": 0.9169807434082031, "entropy_per_token": [0.3057955503463745, 1.0654743909835815, 0.47674161195755005, 1.9541969299316406, 0.6608228087425232, 1.0177578926086426, 1.09443998336792, 0.16398534178733826, 0.16473272442817688, 0.2914188504219055, 0.8694661855697632, 1.738377571105957, 1.506987452507019, 0.20136187970638275, 1.2847659587860107, 0.15182960033416748, 0.8295278549194336, 1.568225383758545, 1.6563894748687744, 1.337317705154419], "max_p": 0.7328373789787292, "max_p_per_token": [0.9353470802307129, 0.7684905529022217, 0.9167540669441223, 0.4357360303401947, 0.8394550681114197, 0.710783064365387, 0.6887586116790771, 0.9716315269470215, 0.9688267707824707, 0.9472593069076538, 0.7833895087242126, 0.4271871745586395, 0.6147254109382629, 0.9544901251792908, 0.39428088068962097, 0.9738557934761047, 0.7795661687850952, 0.42082783579826355, 0.4204188585281372, 0.7049638032913208], "n_positions_probed": 1, "per_restart_best": [10.81745433807373]}
+{"step": 29, "discrete_loss": 12.543461799621582, "best_sample_loss": 10.786544799804688, "soft_loss": 11.530365943908691, "best_discrete": 10.786544799804688, "best_soft": 11.416769027709961, "best_argmax": 12.05322551727295, "best_sampling": 10.786544799804688, "relax_gap": 0.08076684665659477, "n_match": 3, "g_first_norm": 145.7729949951172, "vocab_size": 50257, "entropy": 0.9035701751708984, "entropy_per_token": [0.3063974678516388, 1.0818743705749512, 0.4845368266105652, 1.8694366216659546, 0.6595085859298706, 1.0099055767059326, 1.0985764265060425, 0.1596170961856842, 0.14982548356056213, 0.3182724714279175, 0.8761851787567139, 1.7339199781417847, 1.4009160995483398, 0.21309390664100647, 1.2334718704223633, 0.14630815386772156, 0.826795220375061, 1.511354684829712, 1.6437116861343384, 1.347693920135498], "max_p": 0.7394258379936218, "max_p_per_token": [0.9355822205543518, 0.7636668682098389, 0.9149090647697449, 0.4750831127166748, 0.8398173451423645, 0.7093026041984558, 0.6783817410469055, 0.9725840091705322, 0.9724417328834534, 0.9427220821380615, 0.7795865535736084, 0.4257567524909973, 0.6532331705093384, 0.9508374929428101, 0.468199759721756, 0.9751171469688416, 0.7814363837242126, 0.4389244019985199, 0.4092714488506317, 0.7016626000404358], "n_positions_probed": 1, "per_restart_best": [10.786544799804688]}
+{"step": 30, "discrete_loss": 12.543461799621582, "best_sample_loss": 10.781174659729004, "soft_loss": 11.497228622436523, "best_discrete": 10.781174659729004, "best_soft": 11.416769027709961, "best_argmax": 12.05322551727295, "best_sampling": 10.781174659729004, "relax_gap": 0.0834086469826553, "n_match": 3, "g_first_norm": 143.72354125976562, "vocab_size": 50257, "entropy": 0.9139057397842407, "entropy_per_token": [0.31291258335113525, 1.1038495302200317, 0.49118825793266296, 1.9752252101898193, 0.6586005091667175, 1.0025672912597656, 1.098551630973816, 0.15290230512619019, 0.13737058639526367, 0.32412201166152954, 0.982692301273346, 1.739398717880249, 1.4350864887237549, 0.22443810105323792, 1.2465825080871582, 0.14196056127548218, 0.8255438804626465, 1.4568235874176025, 1.653070330619812, 1.3152283430099487], "max_p": 0.730931282043457, "max_p_per_token": [0.9343122839927673, 0.7567143440246582, 0.9133052229881287, 0.4210401773452759, 0.8401029706001282, 0.7076442241668701, 0.6697366237640381, 0.97404545545578, 0.9753589630126953, 0.9414891600608826, 0.759705126285553, 0.4183111786842346, 0.6401639580726624, 0.9471564888954163, 0.3965272307395935, 0.9761174321174622, 0.7827069759368896, 0.4628230333328247, 0.3901246190071106, 0.7112406492233276], "n_positions_probed": 1, "per_restart_best": [10.781174659729004]}
+{"step": 31, "discrete_loss": 12.543461799621582, "best_sample_loss": 10.752859115600586, "soft_loss": 11.46710205078125, "best_discrete": 10.752859115600586, "best_soft": 11.416769027709961, "best_argmax": 12.05322551727295, "best_sampling": 10.752859115600586, "relax_gap": 0.08581042187833698, "n_match": 3, "g_first_norm": 142.6988067626953, "vocab_size": 50257, "entropy": 0.8973173499107361, "entropy_per_token": [0.3157857656478882, 1.1151891946792603, 0.49722254276275635, 1.8914825916290283, 0.6603485345840454, 0.9914913773536682, 1.107804536819458, 0.1483275443315506, 0.12613442540168762, 0.3330361247062683, 0.9728712439537048, 1.6302919387817383, 1.3778250217437744, 0.2369980812072754, 1.1994707584381104, 0.13665390014648438, 0.8283661007881165, 1.41778564453125, 1.6452606916427612, 1.3139989376068115], "max_p": 0.7415555119514465, "max_p_per_token": [0.9339060187339783, 0.7529726624488831, 0.9117902517318726, 0.46172425150871277, 0.8393264412879944, 0.7072756290435791, 0.6549975275993347, 0.9749932885169983, 0.9779134392738342, 0.9395389556884766, 0.7625793814659119, 0.5222927331924438, 0.6600816249847412, 0.9430157542228699, 0.48100897669792175, 0.977290689945221, 0.7825270891189575, 0.45975279808044434, 0.3768197298049927, 0.7113031148910522], "n_positions_probed": 1, "per_restart_best": [10.752859115600586]}
+{"step": 32, "discrete_loss": 12.543461799621582, "best_sample_loss": 10.80884075164795, "soft_loss": 11.459607124328613, "best_discrete": 10.752859115600586, "best_soft": 11.416769027709961, "best_argmax": 12.05322551727295, "best_sampling": 10.752859115600586, "relax_gap": 0.08640793846286254, "n_match": 3, "g_first_norm": 143.59243774414062, "vocab_size": 50257, "entropy": 0.9074921011924744, "entropy_per_token": [0.3206542134284973, 1.147856593132019, 0.5039796829223633, 1.963075041770935, 0.6617670059204102, 0.9814502596855164, 1.1103211641311646, 0.1407063901424408, 0.11625470221042633, 0.3396795392036438, 0.9821155071258545, 1.7072107791900635, 1.3915417194366455, 0.24938470125198364, 1.2219233512878418, 0.13296106457710266, 0.8281070590019226, 1.3550560474395752, 1.6569042205810547, 1.3388926982879639], "max_p": 0.7319414615631104, "max_p_per_token": [0.9329498410224915, 0.7425533533096313, 0.9101368188858032, 0.42575186491012573, 0.8386744260787964, 0.7065593004226685, 0.6428609490394592, 0.9765907526016235, 0.9800925254821777, 0.9380964636802673, 0.7589967846870422, 0.4804266691207886, 0.6576140522956848, 0.93875652551651, 0.3932796120643616, 0.978127121925354, 0.7834554314613342, 0.4941521883010864, 0.35615551471710205, 0.7035991549491882], "n_positions_probed": 1, "per_restart_best": [10.752859115600586]}
+{"step": 33, "discrete_loss": 12.543461799621582, "best_sample_loss": 10.878592491149902, "soft_loss": 11.410575866699219, "best_discrete": 10.752859115600586, "best_soft": 11.410575866699219, "best_argmax": 12.05322551727295, "best_sampling": 10.752859115600586, "relax_gap": 0.09031684801372304, "n_match": 3, "g_first_norm": 142.24588012695312, "vocab_size": 50257, "entropy": 0.8998391032218933, "entropy_per_token": [0.32430022954940796, 1.1674987077713013, 0.5107624530792236, 1.9119318723678589, 0.6657919883728027, 0.9659193754196167, 1.1239955425262451, 0.13576990365982056, 0.1074688732624054, 0.34798991680145264, 0.9889177083969116, 1.737862229347229, 1.358719825744629, 0.267505407333374, 1.17767333984375, 0.12817487120628357, 0.8331394195556641, 1.2858150005340576, 1.6539185047149658, 1.3036270141601562], "max_p": 0.7357141375541687, "max_p_per_token": [0.9322023391723633, 0.7359702587127686, 0.9084160327911377, 0.4504743814468384, 0.8369660973548889, 0.708127498626709, 0.6217602491378784, 0.9775807857513428, 0.9819823503494263, 0.9362516403198242, 0.7560921311378479, 0.45910346508026123, 0.668563187122345, 0.933485209941864, 0.4857405424118042, 0.979158878326416, 0.7825246453285217, 0.5028162598609924, 0.3431938886642456, 0.7138738036155701], "n_positions_probed": 1, "per_restart_best": [10.752859115600586]}
+{"step": 34, "discrete_loss": 12.543461799621582, "best_sample_loss": 10.824786186218262, "soft_loss": 11.37120246887207, "best_discrete": 10.752859115600586, "best_soft": 11.37120246887207, "best_argmax": 12.05322551727295, "best_sampling": 10.752859115600586, "relax_gap": 0.09345580585934236, "n_match": 3, "g_first_norm": 142.73220825195312, "vocab_size": 50257, "entropy": 0.9055193066596985, "entropy_per_token": [0.331127405166626, 1.1937940120697021, 0.5177686214447021, 1.949050784111023, 0.6677843332290649, 0.9511927366256714, 1.1255486011505127, 0.129037007689476, 0.09941216558218002, 0.35269272327423096, 0.9958184957504272, 1.7702405452728271, 1.3551340103149414, 0.28164535760879517, 1.2250876426696777, 0.12496009469032288, 0.8357039093971252, 1.2101256847381592, 1.6708989143371582, 1.3233641386032104], "max_p": 0.7280614972114563, "max_p_per_token": [0.9306222200393677, 0.7268239259719849, 0.906639814376831, 0.4321325123310089, 0.8360295295715332, 0.7094101905822754, 0.6047387719154358, 0.9789486527442932, 0.9836642146110535, 0.9352027773857117, 0.7535282373428345, 0.43559128046035767, 0.6687267422676086, 0.9283038973808289, 0.3950522541999817, 0.9798827767372131, 0.7824084758758545, 0.5438670516014099, 0.3222569227218628, 0.7073997855186462], "n_positions_probed": 1, "per_restart_best": [10.752859115600586]}
+{"step": 35, "discrete_loss": 12.543461799621582, "best_sample_loss": 10.6458158493042, "soft_loss": 11.315042495727539, "best_discrete": 10.6458158493042, "best_soft": 11.315042495727539, "best_argmax": 12.05322551727295, "best_sampling": 10.6458158493042, "relax_gap": 0.09793303663037445, "n_match": 3, "g_first_norm": 144.2600860595703, "vocab_size": 50257, "entropy": 0.8999902606010437, "entropy_per_token": [0.33880460262298584, 1.2134883403778076, 0.5256329774856567, 1.9218690395355225, 0.6715907454490662, 0.9341988563537598, 1.1396063566207886, 0.12485255300998688, 0.09179553389549255, 0.35767871141433716, 1.0006146430969238, 1.7795319557189941, 1.3391671180725098, 0.2970494329929352, 1.194478988647461, 0.1302994191646576, 0.8418375849723816, 1.123731017112732, 1.6701529026031494, 1.3034250736236572], "max_p": 0.7310167551040649, "max_p_per_token": [0.9287088513374329, 0.7196523547172546, 0.904601514339447, 0.44434311985969543, 0.8343285322189331, 0.7116492390632629, 0.5755330324172974, 0.9797556400299072, 0.9852147698402405, 0.9340546131134033, 0.7515066266059875, 0.4237119257450104, 0.6734886169433594, 0.9225387573242188, 0.47837379574775696, 0.9795483350753784, 0.7810105681419373, 0.5650616884231567, 0.3144350051879883, 0.7128174901008606], "n_positions_probed": 1, "per_restart_best": [10.6458158493042]}
+{"step": 36, "discrete_loss": 12.536335945129395, "best_sample_loss": 10.628994941711426, "soft_loss": 11.26957893371582, "best_discrete": 10.628994941711426, "best_soft": 11.26957893371582, "best_argmax": 12.05322551727295, "best_sampling": 10.628994941711426, "relax_gap": 0.10104683034644851, "n_match": 2, "g_first_norm": 142.42050170898438, "vocab_size": 50257, "entropy": 0.9059950113296509, "entropy_per_token": [0.3480740189552307, 1.24021577835083, 0.5313931703567505, 1.9438737630844116, 0.6752176284790039, 0.9187402129173279, 1.1423141956329346, 0.11982684582471848, 0.0851556807756424, 0.3605077862739563, 1.0046117305755615, 1.7924363613128662, 1.3500056266784668, 0.31250083446502686, 1.2214275598526, 0.12683340907096863, 0.8743070960044861, 1.050489902496338, 1.6862457990646362, 1.3357218503952026], "max_p": 0.7233850359916687, "max_p_per_token": [0.9263100028038025, 0.7098175883293152, 0.9030408263206482, 0.4332873523235321, 0.8328301906585693, 0.7131479382514954, 0.5479776859283447, 0.9807511568069458, 0.986534595489502, 0.9333758354187012, 0.7501141428947449, 0.41035425662994385, 0.668543815612793, 0.9164804816246033, 0.39980843663215637, 0.980320930480957, 0.7763307690620422, 0.5920199751853943, 0.30462753772735596, 0.7020278573036194], "n_positions_probed": 1, "per_restart_best": [10.628994941711426]}
+{"step": 37, "discrete_loss": 12.536335945129395, "best_sample_loss": 10.666964530944824, "soft_loss": 11.222432136535645, "best_discrete": 10.628994941711426, "best_soft": 11.222432136535645, "best_argmax": 12.05322551727295, "best_sampling": 10.628994941711426, "relax_gap": 0.10480764190945495, "n_match": 2, "g_first_norm": 141.69468688964844, "vocab_size": 50257, "entropy": 0.9055082201957703, "entropy_per_token": [0.3590313792228699, 1.260290265083313, 0.5354323387145996, 1.928382396697998, 0.6808960437774658, 0.9035571813583374, 1.1512267589569092, 0.1164972111582756, 0.079125314950943, 0.36366403102874756, 1.0052766799926758, 1.788842797279358, 1.3557239770889282, 0.32868337631225586, 1.1983405351638794, 0.1225559338927269, 0.8822652697563171, 1.0424240827560425, 1.6864784955978394, 1.3214712142944336], "max_p": 0.7237882614135742, "max_p_per_token": [0.9233331084251404, 0.7019641995429993, 0.9018464088439941, 0.4395628273487091, 0.8304983377456665, 0.7147535085678101, 0.5084645748138428, 0.9813926815986633, 0.9877094030380249, 0.9326015114784241, 0.7497792840003967, 0.4067244827747345, 0.6655313372612, 0.9099920392036438, 0.46991628408432007, 0.9812281727790833, 0.7741610407829285, 0.5863854885101318, 0.3046521246433258, 0.7052678465843201], "n_positions_probed": 1, "per_restart_best": [10.628994941711426]}
+{"step": 38, "discrete_loss": 12.543461799621582, "best_sample_loss": 10.67243480682373, "soft_loss": 11.1891508102417, "best_discrete": 10.628994941711426, "best_soft": 11.1891508102417, "best_argmax": 12.05322551727295, "best_sampling": 10.628994941711426, "relax_gap": 0.10796947533421279, "n_match": 3, "g_first_norm": 139.88682556152344, "vocab_size": 50257, "entropy": 0.9133480191230774, "entropy_per_token": [0.3702109158039093, 1.285632848739624, 0.5378658175468445, 1.9418559074401855, 0.686693549156189, 0.8889528512954712, 1.1475470066070557, 0.11243518441915512, 0.07395268976688385, 0.36497944593429565, 1.0086278915405273, 1.7906526327133179, 1.384205937385559, 0.3443153202533722, 1.2232091426849365, 0.1192607656121254, 0.8873250484466553, 1.0051631927490234, 1.7359671592712402, 1.3581058979034424], "max_p": 0.7158275246620178, "max_p_per_token": [0.9201914668083191, 0.6919575333595276, 0.9010083675384521, 0.43202653527259827, 0.8281727433204651, 0.716098964214325, 0.4750650227069855, 0.9821948409080505, 0.9886959791183472, 0.9322305917739868, 0.7487417459487915, 0.4013137221336365, 0.6539062857627869, 0.9034360647201538, 0.40275612473487854, 0.9819483757019043, 0.7728970050811768, 0.5966554284095764, 0.2950807809829712, 0.692172110080719], "n_positions_probed": 1, "per_restart_best": [10.628994941711426]}
+{"step": 39, "discrete_loss": 12.46641731262207, "best_sample_loss": 10.681797981262207, "soft_loss": 11.149099349975586, "best_discrete": 10.628994941711426, "best_soft": 11.149099349975586, "best_argmax": 12.05322551727295, "best_sampling": 10.628994941711426, "relax_gap": 0.10566932981721369, "n_match": 3, "g_first_norm": 140.7183837890625, "vocab_size": 50257, "entropy": 0.9135414361953735, "entropy_per_token": [0.38017570972442627, 1.3013485670089722, 0.5393081903457642, 1.9212803840637207, 0.693598747253418, 0.8754246234893799, 1.1458532810211182, 0.10970431566238403, 0.06921914219856262, 0.3664078116416931, 1.0095410346984863, 1.7826931476593018, 1.389689326286316, 0.3598157465457916, 1.2095797061920166, 0.11529971659183502, 0.892679750919342, 0.9837300777435303, 1.7384425401687622, 1.3870370388031006], "max_p": 0.7148742079734802, "max_p_per_token": [0.9172948002815247, 0.6849169135093689, 0.9003620147705078, 0.43978026509284973, 0.8252016305923462, 0.717628538608551, 0.44681477546691895, 0.9827228784561157, 0.9895803332328796, 0.931807816028595, 0.7484365701675415, 0.4022933542728424, 0.6503283381462097, 0.8967922329902649, 0.45674267411231995, 0.9827702045440674, 0.7714919447898865, 0.5916652083396912, 0.298515260219574, 0.6623382568359375], "n_positions_probed": 1, "per_restart_best": [10.628994941711426]}
+{"step": 40, "discrete_loss": 12.46641731262207, "best_sample_loss": 10.591476440429688, "soft_loss": 11.127184867858887, "best_discrete": 10.591476440429688, "best_soft": 11.127184867858887, "best_argmax": 12.05322551727295, "best_sampling": 10.591476440429688, "relax_gap": 0.10742721113685404, "n_match": 3, "g_first_norm": 141.05419921875, "vocab_size": 50257, "entropy": 0.912362277507782, "entropy_per_token": [0.358525812625885, 1.3183717727661133, 0.540341317653656, 1.917873501777649, 0.7010197639465332, 0.8580897450447083, 1.1408238410949707, 0.1067839190363884, 0.06513265520334244, 0.36577892303466797, 1.0109238624572754, 1.780245304107666, 1.4457674026489258, 0.3758409023284912, 1.2308335304260254, 0.11192812025547028, 0.8967087268829346, 0.9588378667831421, 1.7535698413848877, 1.3098481893539429], "max_p": 0.7135743498802185, "max_p_per_token": [0.9245539307594299, 0.6769679188728333, 0.8997933864593506, 0.4381445050239563, 0.8220553398132324, 0.7213050127029419, 0.47578904032707214, 0.9833064675331116, 0.9903318881988525, 0.93183833360672, 0.7480615377426147, 0.40138471126556396, 0.6272845268249512, 0.8896495699882507, 0.4076145589351654, 0.9834790229797363, 0.770401120185852, 0.5964053273200989, 0.29541030526161194, 0.6877104640007019], "n_positions_probed": 1, "per_restart_best": [10.591476440429688]}
+{"step": 41, "discrete_loss": 12.46641731262207, "best_sample_loss": 10.678484916687012, "soft_loss": 11.092565536499023, "best_discrete": 10.591476440429688, "best_soft": 11.092565536499023, "best_argmax": 12.05322551727295, "best_sampling": 10.591476440429688, "relax_gap": 0.11020421839497074, "n_match": 3, "g_first_norm": 141.513671875, "vocab_size": 50257, "entropy": 0.9136198163032532, "entropy_per_token": [0.36799585819244385, 1.3432323932647705, 0.539654552936554, 1.8894656896591187, 0.7053976058959961, 0.8426936864852905, 1.142031192779541, 0.10426491498947144, 0.061262644827365875, 0.3656970262527466, 1.013108491897583, 1.7737641334533691, 1.4656493663787842, 0.39000535011291504, 1.2266967296600342, 0.10823100805282593, 0.8982839584350586, 0.9436997771263123, 1.7610124349594116, 1.3302491903305054], "max_p": 0.7153327465057373, "max_p_per_token": [0.9216903448104858, 0.669623613357544, 0.8996158242225647, 0.44823578000068665, 0.8199000954627991, 0.7248660326004028, 0.4978051483631134, 0.9837995767593384, 0.9910284876823425, 0.9317274689674377, 0.7471725940704346, 0.40433651208877563, 0.6164201498031616, 0.8831893801689148, 0.4424680173397064, 0.9842272400856018, 0.7700693011283875, 0.5916130542755127, 0.299559086561203, 0.6793076992034912], "n_positions_probed": 1, "per_restart_best": [10.591476440429688]}
+{"step": 42, "discrete_loss": 12.456927299499512, "best_sample_loss": 10.590914726257324, "soft_loss": 11.064006805419922, "best_discrete": 10.590914726257324, "best_soft": 11.064006805419922, "best_argmax": 12.05322551727295, "best_sampling": 10.590914726257324, "relax_gap": 0.11181894704768437, "n_match": 4, "g_first_norm": 141.90249633789062, "vocab_size": 50257, "entropy": 0.9441210627555847, "entropy_per_token": [0.37661731243133545, 1.349928855895996, 1.0969237089157104, 1.8849833011627197, 0.7089371681213379, 0.8244121074676514, 1.1416515111923218, 0.10163619369268417, 0.05770254135131836, 0.3648317754268646, 1.0150684118270874, 1.774479866027832, 1.5037040710449219, 0.40390974283218384, 1.245348334312439, 0.10496405512094498, 0.8987834453582764, 0.9269422292709351, 1.777830719947815, 1.3237664699554443], "max_p": 0.6926887631416321, "max_p_per_token": [0.9190240502357483, 0.6645296216011047, 0.4939153492450714, 0.4461306035518646, 0.8181065917015076, 0.7300711274147034, 0.519130289554596, 0.9843244552612305, 0.9916583895683289, 0.931775689125061, 0.7464092373847961, 0.40345728397369385, 0.5973265767097473, 0.8766075968742371, 0.40956956148147583, 0.9848920106887817, 0.7699949741363525, 0.5908495187759399, 0.29692742228507996, 0.6790744066238403], "n_positions_probed": 1, "per_restart_best": [10.590914726257324]}
+{"step": 43, "discrete_loss": 12.54925537109375, "best_sample_loss": 10.729508399963379, "soft_loss": 11.19543170928955, "best_discrete": 10.590914726257324, "best_soft": 11.064006805419922, "best_argmax": 12.05322551727295, "best_sampling": 10.590914726257324, "relax_gap": 0.10788079625207313, "n_match": 4, "g_first_norm": 151.40419006347656, "vocab_size": 50257, "entropy": 0.9581238031387329, "entropy_per_token": [0.379572331905365, 1.2768323421478271, 1.1070928573608398, 2.196535587310791, 0.7029266357421875, 0.8023796081542969, 1.142155647277832, 0.1001393049955368, 0.0529927983880043, 0.36530038714408875, 1.0318949222564697, 1.7736026048660278, 1.498173475265503, 0.4159647226333618, 1.261967658996582, 0.10279256105422974, 0.9030042886734009, 0.9147357940673828, 1.777708649635315, 1.356702208518982], "max_p": 0.6844555139541626, "max_p_per_token": [0.9185042977333069, 0.6872410774230957, 0.47411468625068665, 0.26959332823753357, 0.8201672434806824, 0.7362906336784363, 0.536639928817749, 0.9846225380897522, 0.9924760460853577, 0.9316014647483826, 0.7405202388763428, 0.41078969836235046, 0.5945922136306763, 0.870716392993927, 0.39495643973350525, 0.9853426218032837, 0.7685529589653015, 0.6030731201171875, 0.3052552044391632, 0.664059042930603], "n_positions_probed": 1, "per_restart_best": [10.590914726257324]}
+{"step": 44, "discrete_loss": 12.54925537109375, "best_sample_loss": 10.56567096710205, "soft_loss": 11.32233715057373, "best_discrete": 10.56567096710205, "best_soft": 11.064006805419922, "best_argmax": 12.05322551727295, "best_sampling": 10.56567096710205, "relax_gap": 0.09776820888880243, "n_match": 4, "g_first_norm": 150.7375946044922, "vocab_size": 50257, "entropy": 0.9503812789916992, "entropy_per_token": [0.3753398060798645, 1.2088546752929688, 1.099919319152832, 2.1218791007995605, 0.6969287395477295, 0.7716615200042725, 1.1487419605255127, 0.09890662878751755, 0.048291295766830444, 0.3704312741756439, 1.0700451135635376, 1.7657239437103271, 1.4811224937438965, 0.43429088592529297, 1.2812304496765137, 0.10058000683784485, 0.9105071425437927, 0.9031122922897339, 1.7853972911834717, 1.3346619606018066], "max_p": 0.686736524105072, "max_p_per_token": [0.9198400378227234, 0.7069903016090393, 0.46709418296813965, 0.2932133674621582, 0.822531521320343, 0.74680095911026, 0.5455247759819031, 0.9848687648773193, 0.9932746887207031, 0.9303267002105713, 0.7274194359779358, 0.4162193834781647, 0.5974172949790955, 0.8614310026168823, 0.38061121106147766, 0.9857973456382751, 0.7657530307769775, 0.6151872873306274, 0.3074524998664856, 0.6669762134552002], "n_positions_probed": 1, "per_restart_best": [10.56567096710205]}
+{"step": 45, "discrete_loss": 12.54925537109375, "best_sample_loss": 10.656002044677734, "soft_loss": 11.266387939453125, "best_discrete": 10.56567096710205, "best_soft": 11.064006805419922, "best_argmax": 12.05322551727295, "best_sampling": 10.56567096710205, "relax_gap": 0.1022265778888851, "n_match": 4, "g_first_norm": 150.7569122314453, "vocab_size": 50257, "entropy": 0.9455680847167969, "entropy_per_token": [0.3741835653781891, 1.1653436422348022, 1.0951975584030151, 2.0318217277526855, 0.6902309656143188, 0.7327837944030762, 1.1523990631103516, 0.09773522615432739, 0.04422314465045929, 0.374173104763031, 1.0944931507110596, 1.7599050998687744, 1.5089621543884277, 0.45324862003326416, 1.3023806810379028, 0.09843862056732178, 0.9172132611274719, 0.8946026563644409, 1.798435926437378, 1.325589656829834], "max_p": 0.6879014372825623, "max_p_per_token": [0.9203135371208191, 0.7188832759857178, 0.45609545707702637, 0.3190605342388153, 0.8247156143188477, 0.7598711848258972, 0.5559784770011902, 0.9851033091545105, 0.9939508438110352, 0.929355800151825, 0.7192360758781433, 0.4193398654460907, 0.582271158695221, 0.8513897061347961, 0.37664419412612915, 0.9862290620803833, 0.763014018535614, 0.6234496831893921, 0.30909955501556396, 0.6640263795852661], "n_positions_probed": 1, "per_restart_best": [10.56567096710205]}
+{"step": 46, "discrete_loss": 12.54925537109375, "best_sample_loss": 10.506983757019043, "soft_loss": 11.205240249633789, "best_discrete": 10.506983757019043, "best_soft": 11.064006805419922, "best_argmax": 12.05322551727295, "best_sampling": 10.506983757019043, "relax_gap": 0.1070991928776744, "n_match": 5, "g_first_norm": 151.7391815185547, "vocab_size": 50257, "entropy": 0.8823925256729126, "entropy_per_token": [0.3724031448364258, 1.1334882974624634, 1.0927659273147583, 1.9222567081451416, 0.6802676916122437, 0.7019696235656738, 0.015441606752574444, 0.09663309156894684, 0.04060230404138565, 0.37860655784606934, 1.113128423690796, 1.761553168296814, 1.481074333190918, 0.4730570316314697, 1.3287301063537598, 0.09661514312028885, 0.9226405620574951, 0.8890791535377502, 1.8167475461959839, 1.3307888507843018], "max_p": 0.7097718119621277, "max_p_per_token": [0.9210267066955566, 0.7271744012832642, 0.43859103322029114, 0.3455730080604553, 0.8281720876693726, 0.7710258960723877, 0.998151957988739, 0.9853282570838928, 0.9945390820503235, 0.9281901717185974, 0.7131767272949219, 0.41859835386276245, 0.5884401202201843, 0.8403879404067993, 0.3612455725669861, 0.9866008162498474, 0.7604489922523499, 0.628164529800415, 0.30643510818481445, 0.6541663408279419], "n_positions_probed": 1, "per_restart_best": [10.506983757019043]}
+{"step": 47, "discrete_loss": 12.501662254333496, "best_sample_loss": 10.596360206604004, "soft_loss": 11.12341594696045, "best_discrete": 10.506983757019043, "best_soft": 11.064006805419922, "best_argmax": 12.05322551727295, "best_sampling": 10.506983757019043, "relax_gap": 0.11024504416565088, "n_match": 4, "g_first_norm": 156.78578186035156, "vocab_size": 50257, "entropy": 0.8881444931030273, "entropy_per_token": [0.3723354935646057, 1.1141647100448608, 1.0899888277053833, 1.7813668251037598, 0.6695704460144043, 0.6716628670692444, 0.015405474230647087, 0.2967463731765747, 0.03762578219175339, 0.38217130303382874, 1.119781255722046, 1.767082691192627, 1.506317377090454, 0.4922787547111511, 1.356888771057129, 0.09440663456916809, 0.9284859895706177, 0.8828378319740295, 1.8463356494903564, 1.3374354839324951], "max_p": 0.7082595229148865, "max_p_per_token": [0.9212630987167358, 0.731633722782135, 0.45951956510543823, 0.3736662268638611, 0.831911027431488, 0.7831335067749023, 0.9981574416160583, 0.936141848564148, 0.9950120449066162, 0.9271533489227295, 0.7121668457984924, 0.4143180549144745, 0.572593092918396, 0.8291409611701965, 0.3584413230419159, 0.9870303273200989, 0.757362425327301, 0.6340968608856201, 0.30102139711380005, 0.6414266228675842], "n_positions_probed": 1, "per_restart_best": [10.506983757019043]}
+{"step": 48, "discrete_loss": 12.501662254333496, "best_sample_loss": 10.506220817565918, "soft_loss": 11.027050971984863, "best_discrete": 10.506220817565918, "best_soft": 11.027050971984863, "best_argmax": 12.05322551727295, "best_sampling": 10.506220817565918, "relax_gap": 0.11795321712818493, "n_match": 4, "g_first_norm": 158.88131713867188, "vocab_size": 50257, "entropy": 0.8827149271965027, "entropy_per_token": [0.3733738660812378, 1.1052900552749634, 1.0890482664108276, 1.62577486038208, 0.6590087413787842, 0.651921272277832, 0.015273073688149452, 0.2980908155441284, 0.03528665751218796, 0.38584667444229126, 1.1264042854309082, 1.7804901599884033, 1.481489658355713, 0.5128939151763916, 1.3890022039413452, 0.09240181744098663, 0.9331647157669067, 0.8782531023025513, 1.876497745513916, 1.3447859287261963], "max_p": 0.7076323628425598, "max_p_per_token": [0.9211278557777405, 0.7329900860786438, 0.4849012792110443, 0.3901168406009674, 0.8356513977050781, 0.7908530831336975, 0.9981764554977417, 0.935206413269043, 0.9954004883766174, 0.9260391592979431, 0.7112573385238647, 0.40583106875419617, 0.5757960081100464, 0.8164377212524414, 0.3382868766784668, 0.9874199032783508, 0.7543894648551941, 0.637275218963623, 0.28992873430252075, 0.6255613565444946], "n_positions_probed": 1, "per_restart_best": [10.506220817565918]}
+{"step": 49, "discrete_loss": 12.46641731262207, "best_sample_loss": 10.525437355041504, "soft_loss": 10.929131507873535, "best_discrete": 10.506220817565918, "best_soft": 10.929131507873535, "best_argmax": 12.05322551727295, "best_sampling": 10.506220817565918, "relax_gap": 0.12331416205617111, "n_match": 4, "g_first_norm": 155.9275360107422, "vocab_size": 50257, "entropy": 0.8861484527587891, "entropy_per_token": [0.3808494806289673, 1.102325677871704, 1.0920829772949219, 1.5129631757736206, 0.654529333114624, 0.6477334499359131, 0.014995129778981209, 0.3003728687763214, 0.03292408585548401, 0.45809614658355713, 1.1241402626037598, 1.7864214181900024, 1.5312694311141968, 0.532717764377594, 1.4160046577453613, 0.08987794816493988, 0.9387626647949219, 0.8759298324584961, 1.9048371315002441, 1.3261359930038452], "max_p": 0.7052310109138489, "max_p_per_token": [0.9189823865890503, 0.7325953245162964, 0.5040721893310547, 0.40999144315719604, 0.8371894359588623, 0.7922059893608093, 0.9982157945632935, 0.9339882731437683, 0.995762050151825, 0.9073809385299683, 0.7138552665710449, 0.40189021825790405, 0.5444106459617615, 0.8035646080970764, 0.3372306525707245, 0.9878841042518616, 0.7507739663124084, 0.636307954788208, 0.2791339159011841, 0.6191843152046204], "n_positions_probed": 1, "per_restart_best": [10.506220817565918]}
+{"step": 50, "discrete_loss": 12.478492736816406, "best_sample_loss": 10.506426811218262, "soft_loss": 10.871078491210938, "best_discrete": 10.506220817565918, "best_soft": 10.871078491210938, "best_argmax": 12.05322551727295, "best_sampling": 10.506220817565918, "relax_gap": 0.12881477591143453, "n_match": 3, "g_first_norm": 150.93597412109375, "vocab_size": 50257, "entropy": 0.8945521712303162, "entropy_per_token": [0.3919634222984314, 1.0955674648284912, 1.0982234477996826, 1.5500667095184326, 0.651071310043335, 0.6498470902442932, 0.014689898118376732, 0.30408281087875366, 0.030703788623213768, 0.4551185965538025, 1.1996023654937744, 1.7901443243026733, 1.5149474143981934, 0.5530570149421692, 1.4431822299957275, 0.0876939594745636, 0.9434424638748169, 0.8713135719299316, 1.9339299201965332, 1.3123953342437744], "max_p": 0.7012446522712708, "max_p_per_token": [0.9158880710601807, 0.7330538630485535, 0.5142404437065125, 0.39759010076522827, 0.8384385704994202, 0.7909451127052307, 0.9982587695121765, 0.9323303699493408, 0.9960951209068298, 0.9080132842063904, 0.7024339437484741, 0.40257760882377625, 0.5400246977806091, 0.7895029783248901, 0.3172069191932678, 0.9882882833480835, 0.7472104430198669, 0.6392099857330322, 0.2636195719242096, 0.6099640130996704], "n_positions_probed": 1, "per_restart_best": [10.506220817565918]}
+{"step": 51, "discrete_loss": 12.478492736816406, "best_sample_loss": 10.46690559387207, "soft_loss": 10.82214069366455, "best_discrete": 10.46690559387207, "best_soft": 10.82214069366455, "best_argmax": 12.05322551727295, "best_sampling": 10.46690559387207, "relax_gap": 0.13273654744093993, "n_match": 3, "g_first_norm": 153.5834197998047, "vocab_size": 50257, "entropy": 0.8978933691978455, "entropy_per_token": [0.4030472934246063, 1.0886057615280151, 1.100257158279419, 1.5746887922286987, 0.6479532718658447, 0.6453027129173279, 0.014328807592391968, 0.30938535928726196, 0.02863115817308426, 0.45327484607696533, 1.2000539302825928, 1.776186466217041, 1.5119664669036865, 0.5748699903488159, 1.4679687023162842, 0.08565863966941833, 0.9468281269073486, 0.8681474924087524, 1.9619028568267822, 1.2988094091415405], "max_p": 0.6990190744400024, "max_p_per_token": [0.9128009676933289, 0.7335832715034485, 0.5248242616653442, 0.39088112115859985, 0.8395574688911438, 0.7925575375556946, 0.9983093738555908, 0.9301386475563049, 0.996402382850647, 0.908291220664978, 0.7039265632629395, 0.40820640325546265, 0.5279673337936401, 0.7733622193336487, 0.30365464091300964, 0.9886593818664551, 0.7437047958374023, 0.6396799087524414, 0.26491403579711914, 0.5989592671394348], "n_positions_probed": 1, "per_restart_best": [10.46690559387207]}
+{"step": 52, "discrete_loss": 12.504674911499023, "best_sample_loss": 10.494145393371582, "soft_loss": 10.772439002990723, "best_discrete": 10.46690559387207, "best_soft": 10.772439002990723, "best_argmax": 12.05322551727295, "best_sampling": 10.46690559387207, "relax_gap": 0.1385270645393088, "n_match": 3, "g_first_norm": 155.3462677001953, "vocab_size": 50257, "entropy": 0.904232919216156, "entropy_per_token": [0.41275566816329956, 1.0835309028625488, 1.098482608795166, 1.5795397758483887, 0.6444611549377441, 0.6379801034927368, 0.013947508297860622, 0.31604841351509094, 0.026684734970331192, 0.4535396695137024, 1.2068642377853394, 1.7809019088745117, 1.5605645179748535, 0.5982024669647217, 1.495010495185852, 0.08390185236930847, 0.9491399526596069, 0.8634868860244751, 1.988593339920044, 1.291022539138794], "max_p": 0.6914151906967163, "max_p_per_token": [0.9100296497344971, 0.7334746718406677, 0.5366493463516235, 0.39153674244880676, 0.8408259749412537, 0.7954319715499878, 0.9983624815940857, 0.9274588823318481, 0.9966874718666077, 0.9079555869102478, 0.7031581401824951, 0.40881288051605225, 0.4152352809906006, 0.7546430230140686, 0.2876540720462799, 0.9889788031578064, 0.7401427626609802, 0.6424416303634644, 0.2653305232524872, 0.5834939479827881], "n_positions_probed": 1, "per_restart_best": [10.46690559387207]}
+{"step": 53, "discrete_loss": 12.504674911499023, "best_sample_loss": 10.626564025878906, "soft_loss": 10.804758071899414, "best_discrete": 10.46690559387207, "best_soft": 10.772439002990723, "best_argmax": 12.05322551727295, "best_sampling": 10.46690559387207, "relax_gap": 0.13594250563334545, "n_match": 3, "g_first_norm": 173.2002716064453, "vocab_size": 50257, "entropy": 0.9086498618125916, "entropy_per_token": [0.4312666952610016, 1.0746983289718628, 1.1020703315734863, 1.5761607885360718, 0.6492393016815186, 0.627324640750885, 0.013610223308205605, 0.32794880867004395, 0.025091134011745453, 0.44990143179893494, 1.2065967321395874, 1.7826426029205322, 1.5856335163116455, 0.6365712881088257, 1.5289390087127686, 0.08292286098003387, 0.9480139017105103, 0.8626010417938232, 1.9881303310394287, 1.273632526397705], "max_p": 0.6876806616783142, "max_p_per_token": [0.9045709371566772, 0.7339586019515991, 0.5422132015228271, 0.38822829723358154, 0.8394660353660583, 0.7997521758079529, 0.9984098672866821, 0.9230602383613586, 0.996918797492981, 0.908531665802002, 0.7038795948028564, 0.4093471169471741, 0.37658098340034485, 0.7183493971824646, 0.2848183512687683, 0.9891760945320129, 0.7377936840057373, 0.6399328708648682, 0.29037582874298096, 0.5682481527328491], "n_positions_probed": 1, "per_restart_best": [10.46690559387207]}
+{"step": 54, "discrete_loss": 12.577956199645996, "best_sample_loss": 10.604941368103027, "soft_loss": 10.706880569458008, "best_discrete": 10.46690559387207, "best_soft": 10.706880569458008, "best_argmax": 12.05322551727295, "best_sampling": 10.46690559387207, "relax_gap": 0.14875831975314474, "n_match": 3, "g_first_norm": 172.1678466796875, "vocab_size": 50257, "entropy": 0.9219194650650024, "entropy_per_token": [0.4479588270187378, 1.0679314136505127, 1.103556513786316, 1.5786974430084229, 0.6535131931304932, 0.6217406988143921, 0.013356766663491726, 0.33873486518859863, 0.023535801097750664, 0.44789621233940125, 1.2105369567871094, 1.7957470417022705, 1.5983680486679077, 0.680088460445404, 1.6961634159088135, 0.08201560378074646, 0.9536374807357788, 0.8529878854751587, 1.9999433755874634, 1.271978735923767], "max_p": 0.6869795322418213, "max_p_per_token": [0.8994969725608826, 0.7337064146995544, 0.5481552481651306, 0.38032862544059753, 0.8382608294487, 0.8020473122596741, 0.9984459280967712, 0.9188231229782104, 0.9971415400505066, 0.9086189270019531, 0.7031266093254089, 0.40379881858825684, 0.34213849902153015, 0.6686489582061768, 0.39151865243911743, 0.9893553853034973, 0.7318546772003174, 0.6522934436798096, 0.2878424823284149, 0.543988049030304], "n_positions_probed": 1, "per_restart_best": [10.46690559387207]}
+{"step": 55, "discrete_loss": 12.600129127502441, "best_sample_loss": 10.46690559387207, "soft_loss": 10.408526420593262, "best_discrete": 10.46690559387207, "best_soft": 10.408526420593262, "best_argmax": 12.05322551727295, "best_sampling": 10.46690559387207, "relax_gap": 0.1739349402479967, "n_match": 4, "g_first_norm": 177.87904357910156, "vocab_size": 50257, "entropy": 0.9757288098335266, "entropy_per_token": [0.46171021461486816, 1.0396398305892944, 1.1105117797851562, 1.640779733657837, 0.6553652286529541, 0.6166021823883057, 0.01393603440374136, 0.34056487679481506, 0.02265210822224617, 0.4610312581062317, 1.2779552936553955, 1.8832225799560547, 1.6259880065917969, 0.6805797219276428, 1.8104639053344727, 0.7233003973960876, 0.9804046750068665, 0.8447811603546143, 2.048825740814209, 1.2762614488601685], "max_p": 0.6511661410331726, "max_p_per_token": [0.8955255150794983, 0.7407145500183105, 0.545957088470459, 0.38053715229034424, 0.8383859992027283, 0.8036983013153076, 0.9983689188957214, 0.9174885749816895, 0.9972724318504333, 0.9047885537147522, 0.6757758259773254, 0.3503696620464325, 0.32730382680892944, 0.6717464327812195, 0.2836231589317322, 0.5289591550827026, 0.7153456211090088, 0.6583145260810852, 0.2790125608444214, 0.5101343989372253], "n_positions_probed": 1, "per_restart_best": [10.46690559387207]}
+{"step": 56, "discrete_loss": 12.47325611114502, "best_sample_loss": 10.414631843566895, "soft_loss": 10.160301208496094, "best_discrete": 10.414631843566895, "best_soft": 10.160301208496094, "best_argmax": 12.05322551727295, "best_sampling": 10.414631843566895, "relax_gap": 0.18543312845009813, "n_match": 4, "g_first_norm": 193.02777099609375, "vocab_size": 50257, "entropy": 0.9758685231208801, "entropy_per_token": [0.46388283371925354, 0.9979629516601562, 1.10502028465271, 1.587158441543579, 0.6435847878456116, 0.5895947813987732, 0.014138005673885345, 0.34690240025520325, 0.021522749215364456, 0.47576481103897095, 1.2434015274047852, 1.9061881303787231, 1.6649068593978882, 0.6215158104896545, 1.8226802349090576, 0.7164555191993713, 1.1621237993240356, 0.8268107175827026, 2.043954849243164, 1.2638009786605835], "max_p": 0.6557343006134033, "max_p_per_token": [0.8955326080322266, 0.7525264620780945, 0.5546022057533264, 0.37646785378456116, 0.8431047201156616, 0.8149142265319824, 0.9983404874801636, 0.9144887328147888, 0.997433602809906, 0.9001694321632385, 0.684815526008606, 0.33535897731781006, 0.31678473949432373, 0.7384409308433533, 0.30770906805992126, 0.5523447394371033, 0.6712814569473267, 0.6679850220680237, 0.2960827946662903, 0.4963022470474243], "n_positions_probed": 1, "per_restart_best": [10.414631843566895]}
+{"step": 57, "discrete_loss": 12.47325611114502, "best_sample_loss": 10.379375457763672, "soft_loss": 9.9994478225708, "best_discrete": 10.379375457763672, "best_soft": 9.9994478225708, "best_argmax": 12.05322551727295, "best_sampling": 10.379375457763672, "relax_gap": 0.19832899016350977, "n_match": 4, "g_first_norm": 194.79327392578125, "vocab_size": 50257, "entropy": 0.9791492819786072, "entropy_per_token": [0.46994549036026, 0.963499903678894, 1.1011435985565186, 1.6635143756866455, 0.633994460105896, 0.5843450427055359, 0.014326835051178932, 0.3559269309043884, 0.020386777818202972, 0.47925031185150146, 1.2653894424438477, 1.9382274150848389, 1.705019474029541, 0.6246213912963867, 1.8635203838348389, 0.7045177817344666, 1.0542407035827637, 0.8116051554679871, 2.0623717308044434, 1.2671380043029785], "max_p": 0.652853786945343, "max_p_per_token": [0.8944180011749268, 0.7619722485542297, 0.5626168251037598, 0.3622334599494934, 0.8468884825706482, 0.8170120120048523, 0.998314619064331, 0.9103260636329651, 0.9975937008857727, 0.8985569477081299, 0.6750155091285706, 0.3290482759475708, 0.327025830745697, 0.7348458766937256, 0.22296009957790375, 0.5853822231292725, 0.7044121623039246, 0.665532648563385, 0.2875705063343048, 0.4753497838973999], "n_positions_probed": 1, "per_restart_best": [10.379375457763672]}
+{"step": 58, "discrete_loss": 11.850425720214844, "best_sample_loss": 10.410857200622559, "soft_loss": 9.7826509475708, "best_discrete": 10.379375457763672, "best_soft": 9.7826509475708, "best_argmax": 11.850425720214844, "best_sampling": 10.379375457763672, "relax_gap": 0.174489492737528, "n_match": 4, "g_first_norm": 154.2194366455078, "vocab_size": 50257, "entropy": 0.9789121747016907, "entropy_per_token": [0.48854660987854004, 0.9286866784095764, 1.0745741128921509, 1.6442928314208984, 0.6286804676055908, 0.5867291688919067, 0.014236249029636383, 0.36237451434135437, 0.01911090686917305, 0.48698320984840393, 1.2779327630996704, 1.9525525569915771, 1.7462091445922852, 0.6277523636817932, 1.8476645946502686, 0.6870546936988831, 1.0621922016143799, 0.8073112964630127, 2.0453133583068848, 1.2900445461273193], "max_p": 0.6577550768852234, "max_p_per_token": [0.889403760433197, 0.7722811102867126, 0.590358555316925, 0.38943469524383545, 0.8488168120384216, 0.8158615827560425, 0.9983262419700623, 0.9070061445236206, 0.9977699518203735, 0.8955414891242981, 0.6680420637130737, 0.3292481303215027, 0.3220495879650116, 0.7324156165122986, 0.2620292901992798, 0.6225872039794922, 0.6929517388343811, 0.657170295715332, 0.32047927379608154, 0.44332873821258545], "n_positions_probed": 1, "per_restart_best": [10.379375457763672]}
+{"step": 59, "discrete_loss": 11.850425720214844, "best_sample_loss": 10.379375457763672, "soft_loss": 9.664969444274902, "best_discrete": 10.379375457763672, "best_soft": 9.664969444274902, "best_argmax": 11.850425720214844, "best_sampling": 10.379375457763672, "relax_gap": 0.18442006452240098, "n_match": 4, "g_first_norm": 160.24258422851562, "vocab_size": 50257, "entropy": 0.9396992921829224, "entropy_per_token": [0.5104889273643494, 0.9003381729125977, 1.0467133522033691, 1.7648367881774902, 0.6282116174697876, 0.5962470769882202, 0.014028060249984264, 0.36700862646102905, 0.01796761155128479, 0.4934682846069336, 1.3190356492996216, 1.9565796852111816, 1.7821149826049805, 0.6352487206459045, 1.8431154489517212, 0.6703829169273376, 1.0470725297927856, 0.7967664003372192, 2.075387477874756, 0.32897233963012695], "max_p": 0.6817693114280701, "max_p_per_token": [0.8837399482727051, 0.7803015112876892, 0.6156722903251648, 0.33922460675239563, 0.8490492105484009, 0.8117715120315552, 0.9983539581298828, 0.9043847322463989, 0.9979256391525269, 0.8928115963935852, 0.6485455632209778, 0.32679659128189087, 0.3248971104621887, 0.7264868021011353, 0.2817767560482025, 0.6508780717849731, 0.691365122795105, 0.6537122130393982, 0.32365116477012634, 0.9340407848358154], "n_positions_probed": 1, "per_restart_best": [10.379375457763672]}
+{"step": 60, "discrete_loss": 11.538226127624512, "best_sample_loss": 10.483388900756836, "soft_loss": 10.214024543762207, "best_discrete": 10.379375457763672, "best_soft": 9.664969444274902, "best_argmax": 11.538226127624512, "best_sampling": 10.379375457763672, "relax_gap": 0.1147664787650449, "n_match": 4, "g_first_norm": 177.32582092285156, "vocab_size": 50257, "entropy": 0.9499287009239197, "entropy_per_token": [1.1570430994033813, 0.888344943523407, 0.9693833589553833, 1.6445412635803223, 0.6156629920005798, 0.5855928063392639, 0.01411496289074421, 0.3578605651855469, 0.01698986254632473, 0.509280800819397, 1.2314776182174683, 1.9536573886871338, 1.73463773727417, 0.5909762382507324, 1.801335334777832, 0.6545299887657166, 1.0215450525283813, 0.7846329212188721, 2.1188416481018066, 0.34812530875205994], "max_p": 0.6851648688316345, "max_p_per_token": [0.6562146544456482, 0.7838066220283508, 0.6673426032066345, 0.4114224314689636, 0.8529561758041382, 0.8158664107322693, 0.99834144115448, 0.9071539044380188, 0.9980543851852417, 0.8869361281394958, 0.6743139624595642, 0.32679569721221924, 0.390603631734848, 0.7648146748542786, 0.31632718443870544, 0.6739187240600586, 0.6988164782524109, 0.6605117917060852, 0.29012781381607056, 0.9289721250534058], "n_positions_probed": 1, "per_restart_best": [10.379375457763672]}
+{"step": 61, "discrete_loss": 11.850425720214844, "best_sample_loss": 10.430473327636719, "soft_loss": 10.064352989196777, "best_discrete": 10.379375457763672, "best_soft": 9.664969444274902, "best_argmax": 11.538226127624512, "best_sampling": 10.379375457763672, "relax_gap": 0.15071802255773184, "n_match": 4, "g_first_norm": 129.21763610839844, "vocab_size": 50257, "entropy": 0.9588947296142578, "entropy_per_token": [1.1504535675048828, 1.0774459838867188, 0.9322003722190857, 1.744110107421875, 0.6034828424453735, 0.5614306330680847, 0.014315472915768623, 0.3546496629714966, 0.016111521050333977, 0.5319777727127075, 1.207154631614685, 1.959216833114624, 1.702967643737793, 0.5724246501922607, 1.7805148363113403, 0.633897602558136, 1.0022305250167847, 0.7739897966384888, 2.1818161010742188, 0.37750327587127686], "max_p": 0.6828598976135254, "max_p_per_token": [0.6589303612709045, 0.7248840928077698, 0.6899533867835999, 0.3418959081172943, 0.8567209243774414, 0.8255206346511841, 0.9983128309249878, 0.9077590703964233, 0.9981690645217896, 0.8789946436882019, 0.6767481565475464, 0.30713382363319397, 0.41913068294525146, 0.7794156074523926, 0.3232506513595581, 0.6996504664421082, 0.7024111747741699, 0.6699740290641785, 0.2776089310646057, 0.9207335114479065], "n_positions_probed": 1, "per_restart_best": [10.379375457763672]}
+{"step": 62, "discrete_loss": 11.761458396911621, "best_sample_loss": 10.399087905883789, "soft_loss": 9.963117599487305, "best_discrete": 10.379375457763672, "best_soft": 9.664969444274902, "best_argmax": 11.538226127624512, "best_sampling": 10.379375457763672, "relax_gap": 0.15290117404968528, "n_match": 4, "g_first_norm": 151.29754638671875, "vocab_size": 50257, "entropy": 0.9442493319511414, "entropy_per_token": [1.1352221965789795, 1.0608954429626465, 0.7915268540382385, 1.687984585762024, 0.5932976007461548, 0.5265297889709473, 0.014233660884201527, 0.34917163848876953, 0.01525479182600975, 0.554785966873169, 1.2172436714172363, 1.965752363204956, 1.674375057220459, 0.5599713921546936, 1.7635552883148193, 0.6185028553009033, 0.9602377414703369, 0.7655407190322876, 2.219040870666504, 0.4118640124797821], "max_p": 0.6919011473655701, "max_p_per_token": [0.6622490286827087, 0.7312067747116089, 0.7733692526817322, 0.3805946111679077, 0.8597873449325562, 0.8398738503456116, 0.9983236193656921, 0.9091961979866028, 0.9982800483703613, 0.8706380724906921, 0.6662716269493103, 0.3120371997356415, 0.44404280185699463, 0.7886669039726257, 0.3309597373008728, 0.7170543670654297, 0.7168116569519043, 0.6712430715560913, 0.25669729709625244, 0.910719096660614], "n_positions_probed": 1, "per_restart_best": [10.379375457763672]}
+{"step": 63, "discrete_loss": 11.76948070526123, "best_sample_loss": 10.476574897766113, "soft_loss": 9.871763229370117, "best_discrete": 10.379375457763672, "best_soft": 9.664969444274902, "best_argmax": 11.538226127624512, "best_sampling": 10.379375457763672, "relax_gap": 0.1612405443719186, "n_match": 4, "g_first_norm": 129.46084594726562, "vocab_size": 50257, "entropy": 0.955047607421875, "entropy_per_token": [1.1382554769515991, 1.0531837940216064, 0.7844828367233276, 1.8786168098449707, 0.5888694524765015, 0.5011195540428162, 0.014148302376270294, 0.3476284146308899, 0.014520731754601002, 0.574425220489502, 1.229876160621643, 1.9622141122817993, 1.6675169467926025, 0.5528928637504578, 1.7539843320846558, 0.6013479828834534, 0.9599156379699707, 0.7678642272949219, 2.2614283561706543, 0.4486616849899292], "max_p": 0.6868804097175598, "max_p_per_token": [0.6625168323516846, 0.7337145209312439, 0.7762731313705444, 0.2540673613548279, 0.8608969449996948, 0.8500136733055115, 0.9983339905738831, 0.9092273712158203, 0.9983744621276855, 0.8630384802818298, 0.6540036797523499, 0.3257804214954376, 0.4515642523765564, 0.7940794825553894, 0.3332602381706238, 0.7349230051040649, 0.7146158218383789, 0.6589738726615906, 0.26445767283439636, 0.8994932770729065], "n_positions_probed": 1, "per_restart_best": [10.379375457763672]}
+{"step": 64, "discrete_loss": 11.54004192352295, "best_sample_loss": 10.3189697265625, "soft_loss": 9.82851505279541, "best_discrete": 10.3189697265625, "best_soft": 9.664969444274902, "best_argmax": 11.538226127624512, "best_sampling": 10.3189697265625, "relax_gap": 0.14831201498833407, "n_match": 4, "g_first_norm": 151.1428985595703, "vocab_size": 50257, "entropy": 0.9485955238342285, "entropy_per_token": [1.1470311880111694, 1.0406320095062256, 0.7509809136390686, 1.862263560295105, 0.5238969326019287, 0.43955621123313904, 0.01400613784790039, 0.34532347321510315, 0.013692174106836319, 0.5948240756988525, 1.2500097751617432, 1.9576820135116577, 1.6562658548355103, 0.5473992228507996, 1.746434211730957, 0.5874190926551819, 0.9416323900222778, 0.771221399307251, 2.2864246368408203, 0.49521613121032715], "max_p": 0.6867296099662781, "max_p_per_token": [0.6572302579879761, 0.7378756999969482, 0.7904123067855835, 0.2765633463859558, 0.8330380320549011, 0.8745911717414856, 0.998353123664856, 0.9096031785011292, 0.9984797835350037, 0.8547804951667786, 0.6365534663200378, 0.3374790549278259, 0.46316713094711304, 0.7981626391410828, 0.33296075463294983, 0.7485346794128418, 0.7209358215332031, 0.6406717300415039, 0.24054239690303802, 0.88465815782547], "n_positions_probed": 1, "per_restart_best": [10.3189697265625]}
+{"step": 65, "discrete_loss": 11.54004192352295, "best_sample_loss": 10.575390815734863, "soft_loss": 9.851644515991211, "best_discrete": 10.3189697265625, "best_soft": 9.664969444274902, "best_argmax": 11.538226127624512, "best_sampling": 10.3189697265625, "relax_gap": 0.14630773603085, "n_match": 4, "g_first_norm": 129.63552856445312, "vocab_size": 50257, "entropy": 0.962419331073761, "entropy_per_token": [1.1802358627319336, 1.0336750745773315, 0.7337285280227661, 1.8470101356506348, 0.5048139095306396, 0.7179579734802246, 0.013821166940033436, 0.35537758469581604, 0.012720774859189987, 0.6045766472816467, 1.2110284566879272, 1.9565174579620361, 1.6589328050613403, 0.5488868951797485, 1.7370550632476807, 0.5732641220092773, 0.9289938807487488, 0.7855106592178345, 2.285588502883911, 0.558689534664154], "max_p": 0.6814386248588562, "max_p_per_token": [0.6364887356758118, 0.7396072149276733, 0.7978819608688354, 0.3126997649669647, 0.8415513038635254, 0.7879427671432495, 0.998375654220581, 0.9053592681884766, 0.9986017346382141, 0.85030198097229, 0.6428304314613342, 0.32864144444465637, 0.4652940630912781, 0.7976785898208618, 0.3387152850627899, 0.7610903978347778, 0.7263489961624146, 0.5975190997123718, 0.23883730173110962, 0.8630058169364929], "n_positions_probed": 1, "per_restart_best": [10.3189697265625]}
+{"step": 66, "discrete_loss": 11.54004192352295, "best_sample_loss": 10.431553840637207, "soft_loss": 9.746373176574707, "best_discrete": 10.3189697265625, "best_soft": 9.664969444274902, "best_argmax": 11.538226127624512, "best_sampling": 10.3189697265625, "relax_gap": 0.15543000266680748, "n_match": 4, "g_first_norm": 143.36569213867188, "vocab_size": 50257, "entropy": 0.9622383117675781, "entropy_per_token": [1.199456810951233, 1.0234767198562622, 0.7207126617431641, 1.8305504322052002, 0.4980314075946808, 0.6506054401397705, 0.01413761917501688, 0.3636125326156616, 0.011818736791610718, 0.6136122941970825, 1.2005263566970825, 1.9565434455871582, 1.6776684522628784, 0.5446923971176147, 1.736790418624878, 0.5592763423919678, 0.9102271795272827, 0.7883783578872681, 2.296395778656006, 0.6482529044151306], "max_p": 0.6812203526496887, "max_p_per_token": [0.6250309348106384, 0.7432301640510559, 0.8035798668861389, 0.34696295857429504, 0.8443453907966614, 0.8155832290649414, 0.9983553290367126, 0.9016643166542053, 0.9987133741378784, 0.845881462097168, 0.6363900303840637, 0.3299349248409271, 0.4589080214500427, 0.8009459972381592, 0.33637240529060364, 0.7729196548461914, 0.73582524061203, 0.5655876398086548, 0.23426711559295654, 0.8299082517623901], "n_positions_probed": 1, "per_restart_best": [10.3189697265625]}
+{"step": 67, "discrete_loss": 11.54004192352295, "best_sample_loss": 10.47397518157959, "soft_loss": 9.661741256713867, "best_discrete": 10.3189697265625, "best_soft": 9.661741256713867, "best_argmax": 11.538226127624512, "best_sampling": 10.3189697265625, "relax_gap": 0.16276376457354094, "n_match": 4, "g_first_norm": 143.0376739501953, "vocab_size": 50257, "entropy": 0.9851238131523132, "entropy_per_token": [1.2183001041412354, 1.0188014507293701, 0.7166942358016968, 1.860912561416626, 0.4959571361541748, 0.6338290572166443, 0.013966540805995464, 0.6301881074905396, 0.010929328389465809, 0.6172708868980408, 1.1938532590866089, 1.958705186843872, 1.6953179836273193, 0.542631983757019, 1.7383674383163452, 0.5451684594154358, 0.9003664255142212, 0.7870243787765503, 2.305370330810547, 0.8188198804855347], "max_p": 0.6708885431289673, "max_p_per_token": [0.6146737933158875, 0.7448405027389526, 0.8055495619773865, 0.33183372020721436, 0.8449080586433411, 0.8218846917152405, 0.9983748197555542, 0.8232504725456238, 0.9988213181495667, 0.8435177206993103, 0.6283935904502869, 0.32725974917411804, 0.4519640803337097, 0.8026931881904602, 0.33240172266960144, 0.7843509316444397, 0.7418020367622375, 0.5361530780792236, 0.22781649231910706, 0.7572816014289856], "n_positions_probed": 1, "per_restart_best": [10.3189697265625]}
+{"step": 68, "discrete_loss": 11.743057250976562, "best_sample_loss": 10.296488761901855, "soft_loss": 9.523641586303711, "best_discrete": 10.296488761901855, "best_soft": 9.523641586303711, "best_argmax": 11.538226127624512, "best_sampling": 10.296488761901855, "relax_gap": 0.18899811328845248, "n_match": 4, "g_first_norm": 152.231201171875, "vocab_size": 50257, "entropy": 1.0024598836898804, "entropy_per_token": [1.2310302257537842, 1.008379578590393, 0.7058690786361694, 1.8677544593811035, 0.49112647771835327, 0.6307801604270935, 0.013754326850175858, 0.6794542670249939, 0.010746676474809647, 0.6175615787506104, 1.2034271955490112, 1.9771009683609009, 1.752368450164795, 0.5517646670341492, 1.75169038772583, 0.5267930030822754, 0.8900297284126282, 0.7815778851509094, 2.306180477142334, 1.0518074035644531], "max_p": 0.6595847010612488, "max_p_per_token": [0.6052366495132446, 0.748612642288208, 0.81035977602005, 0.33005329966545105, 0.8469733595848083, 0.8226404786109924, 0.9983996748924255, 0.8023995161056519, 0.9988627433776855, 0.8426589965820312, 0.6138545274734497, 0.3181766867637634, 0.4219672977924347, 0.796943187713623, 0.3190856873989105, 0.7982199192047119, 0.7478849291801453, 0.5125951170921326, 0.22787970304489136, 0.628889799118042], "n_positions_probed": 1, "per_restart_best": [10.296488761901855]}
+{"step": 69, "discrete_loss": 11.392038345336914, "best_sample_loss": 10.354233741760254, "soft_loss": 9.358573913574219, "best_discrete": 10.296488761901855, "best_soft": 9.358573913574219, "best_argmax": 11.392038345336914, "best_sampling": 10.296488761901855, "relax_gap": 0.17849873482870168, "n_match": 4, "g_first_norm": 159.7108612060547, "vocab_size": 50257, "entropy": 1.0095478296279907, "entropy_per_token": [1.240548849105835, 0.9909980297088623, 0.6889528036117554, 1.8611228466033936, 0.47771579027175903, 0.6152669191360474, 0.013625586405396461, 0.7038452625274658, 0.009858479723334312, 0.616777777671814, 1.2226848602294922, 2.0026493072509766, 1.8357813358306885, 0.5587595701217651, 1.719064474105835, 0.5173038840293884, 0.8835294842720032, 0.7735721468925476, 2.3230607509613037, 1.1358380317687988], "max_p": 0.6550561785697937, "max_p_per_token": [0.6015182137489319, 0.7551710605621338, 0.8173089623451233, 0.33579346537590027, 0.8532710075378418, 0.8285589814186096, 0.9984138011932373, 0.7912856340408325, 0.9989688396453857, 0.8420102000236511, 0.5905963182449341, 0.3046511709690094, 0.3788152039051056, 0.7923739552497864, 0.37505003809928894, 0.8054680228233337, 0.7510371208190918, 0.4995565116405487, 0.23451077938079834, 0.5467649698257446], "n_positions_probed": 1, "per_restart_best": [10.296488761901855]}
+{"step": 70, "discrete_loss": 11.3493013381958, "best_sample_loss": 10.296488761901855, "soft_loss": 9.25805377960205, "best_discrete": 10.296488761901855, "best_soft": 9.25805377960205, "best_argmax": 11.3493013381958, "best_sampling": 10.296488761901855, "relax_gap": 0.18426222868501224, "n_match": 5, "g_first_norm": 141.3167266845703, "vocab_size": 50257, "entropy": 0.9761790633201599, "entropy_per_token": [1.2503656148910522, 0.9773411750793457, 0.6756386756896973, 1.882200002670288, 0.4713083505630493, 0.6154210567474365, 0.013617640361189842, 0.7360186576843262, 0.009094709530472755, 0.6171332597732544, 0.44802290201187134, 2.01023530960083, 1.8600208759307861, 0.5690112113952637, 1.7358708381652832, 0.5002359747886658, 0.8923330903053284, 0.767436146736145, 2.330745220184326, 1.1615300178527832], "max_p": 0.6656265258789062, "max_p_per_token": [0.598580002784729, 0.7601687908172607, 0.8229717016220093, 0.31734228134155273, 0.8561009764671326, 0.8284028172492981, 0.9984112977981567, 0.7760359644889832, 0.9990596175193787, 0.8406534790992737, 0.8985651731491089, 0.2890649735927582, 0.3713410496711731, 0.7857186198234558, 0.35575932264328003, 0.8171474933624268, 0.748837411403656, 0.5017192959785461, 0.25854089856147766, 0.4881097972393036], "n_positions_probed": 1, "per_restart_best": [10.296488761901855]}
+{"step": 71, "discrete_loss": 11.121016502380371, "best_sample_loss": 10.349366188049316, "soft_loss": 9.359999656677246, "best_discrete": 10.296488761901855, "best_soft": 9.25805377960205, "best_argmax": 11.121016502380371, "best_sampling": 10.296488761901855, "relax_gap": 0.15835034911837353, "n_match": 5, "g_first_norm": 151.24794006347656, "vocab_size": 50257, "entropy": 0.8939010500907898, "entropy_per_token": [1.2616435289382935, 0.9897022843360901, 0.6569344997406006, 1.885331392288208, 0.4644924998283386, 0.6301029920578003, 0.013562537729740143, 0.7156341075897217, 0.008434491232037544, 0.6176261901855469, 0.45335692167282104, 0.3331094980239868, 1.8819231986999512, 0.5882792472839355, 1.7509214878082275, 0.4877850413322449, 0.8969168066978455, 0.7599245309829712, 2.3427789211273193, 1.1395610570907593], "max_p": 0.6965687274932861, "max_p_per_token": [0.5973232984542847, 0.7568985223770142, 0.8306129574775696, 0.316650390625, 0.8593747019767761, 0.8222532868385315, 0.9984153509140015, 0.783905565738678, 0.9991366267204285, 0.8395940065383911, 0.8972763419151306, 0.942633867263794, 0.3613438606262207, 0.7705768346786499, 0.3430432975292206, 0.8252496123313904, 0.7480162382125854, 0.5084774494171143, 0.23152808845043182, 0.4990639388561249], "n_positions_probed": 1, "per_restart_best": [10.296488761901855]}
+{"step": 72, "discrete_loss": 11.286331176757812, "best_sample_loss": 10.323390007019043, "soft_loss": 9.339836120605469, "best_discrete": 10.296488761901855, "best_soft": 9.25805377960205, "best_argmax": 11.121016502380371, "best_sampling": 10.296488761901855, "relax_gap": 0.1724648183424569, "n_match": 5, "g_first_norm": 218.06491088867188, "vocab_size": 50257, "entropy": 0.9046701788902283, "entropy_per_token": [1.2709977626800537, 1.010878324508667, 0.6473768949508667, 1.8866214752197266, 0.46882906556129456, 0.6350850462913513, 0.013090893626213074, 0.7527559995651245, 0.007671665400266647, 0.6021516919136047, 0.42596355080604553, 0.3402760922908783, 2.126199722290039, 0.5876544117927551, 1.7857745885849, 0.4837340712547302, 0.8772751092910767, 0.7460122108459473, 2.3085365295410156, 1.1165173053741455], "max_p": 0.6951954960823059, "max_p_per_token": [0.5901773571968079, 0.7500911951065063, 0.8346058130264282, 0.306162029504776, 0.8577925562858582, 0.8203664422035217, 0.9984788298606873, 0.766520619392395, 0.999225378036499, 0.8458209037780762, 0.9056118130683899, 0.9411706924438477, 0.24931678175926208, 0.7728089690208435, 0.3290928900241852, 0.8278250098228455, 0.7571088075637817, 0.5422322154045105, 0.31026574969291687, 0.499235600233078], "n_positions_probed": 1, "per_restart_best": [10.296488761901855]}
+{"step": 73, "discrete_loss": 11.461796760559082, "best_sample_loss": 10.394312858581543, "soft_loss": 9.187520980834961, "best_discrete": 10.296488761901855, "best_soft": 9.187520980834961, "best_argmax": 11.121016502380371, "best_sampling": 10.296488761901855, "relax_gap": 0.19842227420661285, "n_match": 5, "g_first_norm": 160.85098266601562, "vocab_size": 50257, "entropy": 0.9220271110534668, "entropy_per_token": [1.2760577201843262, 1.0282683372497559, 0.6366766691207886, 1.8648171424865723, 0.47147905826568604, 0.6212607026100159, 0.013077112846076488, 0.8118078112602234, 0.007243641186505556, 0.6051648259162903, 0.4069334864616394, 0.34628552198410034, 2.1841654777526855, 0.7241789102554321, 1.803016185760498, 0.46744057536125183, 0.9057148694992065, 0.7437225580215454, 2.388223648071289, 1.1350067853927612], "max_p": 0.6888495683670044, "max_p_per_token": [0.5901413559913635, 0.7452126145362854, 0.8389166593551636, 0.32455649971961975, 0.8568053841590881, 0.8257351517677307, 0.9984776377677917, 0.7371923923492432, 0.9992750287055969, 0.8448069095611572, 0.9113276600837708, 0.9398989081382751, 0.2706765830516815, 0.7528479099273682, 0.29449543356895447, 0.8376923203468323, 0.7448644042015076, 0.5377619862556458, 0.23948439955711365, 0.4868226647377014], "n_positions_probed": 1, "per_restart_best": [10.296488761901855]}
+{"step": 74, "discrete_loss": 11.663164138793945, "best_sample_loss": 10.546177864074707, "soft_loss": 9.122784614562988, "best_discrete": 10.296488761901855, "best_soft": 9.122784614562988, "best_argmax": 11.121016502380371, "best_sampling": 10.296488761901855, "relax_gap": 0.2178122072192367, "n_match": 5, "g_first_norm": 207.8253631591797, "vocab_size": 50257, "entropy": 0.9105059504508972, "entropy_per_token": [1.2683274745941162, 1.0604184865951538, 0.6191834211349487, 1.8672890663146973, 0.4639297127723694, 0.6344842910766602, 0.01278744637966156, 0.7911120653152466, 0.006589858792722225, 0.5864661931991577, 0.38318660855293274, 0.35637450218200684, 2.270510196685791, 0.7451859712600708, 1.563044548034668, 0.46715977787971497, 0.8793962001800537, 0.7303979396820068, 2.380166530609131, 1.1241081953048706], "max_p": 0.7038642764091492, "max_p_per_token": [0.5982187390327454, 0.7347107529640198, 0.8454958200454712, 0.31946906447410583, 0.8609023690223694, 0.8206526637077332, 0.9985176920890808, 0.7470459342002869, 0.9993496537208557, 0.8518591523170471, 0.9183253049850464, 0.9378764629364014, 0.25835147500038147, 0.7411471009254456, 0.5054948329925537, 0.8380971550941467, 0.7557947039604187, 0.5649757385253906, 0.29827114939689636, 0.4827287793159485], "n_positions_probed": 1, "per_restart_best": [10.296488761901855]}
+{"step": 75, "discrete_loss": 11.693305015563965, "best_sample_loss": 10.30085563659668, "soft_loss": 9.733757019042969, "best_discrete": 10.296488761901855, "best_soft": 9.122784614562988, "best_argmax": 11.121016502380371, "best_sampling": 10.296488761901855, "relax_gap": 0.16757862673665044, "n_match": 5, "g_first_norm": 171.1571807861328, "vocab_size": 50257, "entropy": 0.9301714301109314, "entropy_per_token": [1.2539128065109253, 1.1069408655166626, 0.612275242805481, 1.8659900426864624, 0.4785706400871277, 0.63133704662323, 0.012577492743730545, 0.845470666885376, 0.006245839409530163, 0.5804815292358398, 0.36611220240592957, 0.3615780472755432, 2.2826128005981445, 0.7681758403778076, 1.691718339920044, 0.46321040391921997, 0.8959336280822754, 0.7300925254821777, 2.492563247680664, 1.1576306819915771], "max_p": 0.692691445350647, "max_p_per_token": [0.6045249104499817, 0.7180789113044739, 0.8486408591270447, 0.3277081251144409, 0.8542165160179138, 0.8216139078140259, 0.9985471367835999, 0.7172642350196838, 0.9993889331817627, 0.8541475534439087, 0.9230806231498718, 0.9368156790733337, 0.28614312410354614, 0.728579044342041, 0.37894922494888306, 0.8451816439628601, 0.7491632103919983, 0.56052166223526, 0.23400798439979553, 0.4672555923461914], "n_positions_probed": 1, "per_restart_best": [10.296488761901855]}
+{"step": 76, "discrete_loss": 11.301119804382324, "best_sample_loss": 10.469897270202637, "soft_loss": 9.390493392944336, "best_discrete": 10.296488761901855, "best_soft": 9.122784614562988, "best_argmax": 11.121016502380371, "best_sampling": 10.296488761901855, "relax_gap": 0.169065229332149, "n_match": 4, "g_first_norm": 227.8977508544922, "vocab_size": 50257, "entropy": 0.9342323541641235, "entropy_per_token": [1.219305157661438, 1.1835477352142334, 0.5984616279602051, 1.8750725984573364, 0.4915269613265991, 0.6620566844940186, 0.01175273023545742, 0.8614299297332764, 0.005798459053039551, 0.5601856708526611, 0.3450223207473755, 0.37891775369644165, 2.3524911403656006, 0.8177732229232788, 1.8041924238204956, 0.4548616409301758, 0.6622978448867798, 0.7214409112930298, 2.51019287109375, 1.168318510055542], "max_p": 0.6917834877967834, "max_p_per_token": [0.6171658039093018, 0.6889519095420837, 0.8540924191474915, 0.32067686319351196, 0.8483873605728149, 0.8089316487312317, 0.9986604452133179, 0.7068421840667725, 0.9994396567344666, 0.8621508479118347, 0.9287015199661255, 0.9331274032592773, 0.2686123847961426, 0.7034876942634583, 0.35737380385398865, 0.8500903844833374, 0.8568668365478516, 0.5734667181968689, 0.21130460500717163, 0.44733908772468567], "n_positions_probed": 1, "per_restart_best": [10.296488761901855]}
+{"step": 77, "discrete_loss": 11.545123100280762, "best_sample_loss": 10.46948528289795, "soft_loss": 8.97346305847168, "best_discrete": 10.296488761901855, "best_soft": 8.97346305847168, "best_argmax": 11.121016502380371, "best_sampling": 10.296488761901855, "relax_gap": 0.22274860297908325, "n_match": 4, "g_first_norm": 157.92054748535156, "vocab_size": 50257, "entropy": 0.9424070715904236, "entropy_per_token": [1.2291803359985352, 1.241171956062317, 0.5774577260017395, 1.8773725032806396, 0.4997525215148926, 0.655966579914093, 0.011441759765148163, 0.8667263984680176, 0.005481393076479435, 0.5557395219802856, 0.3321324288845062, 0.3829652667045593, 2.344707489013672, 0.8408234119415283, 1.836742877960205, 0.4389934241771698, 0.6923989057540894, 0.7312926650047302, 2.5347962379455566, 1.1929980516433716], "max_p": 0.6919158101081848, "max_p_per_token": [0.6153489947319031, 0.6680380702018738, 0.8619419932365417, 0.32274219393730164, 0.8447726964950562, 0.8112091422080994, 0.9987006187438965, 0.7029414772987366, 0.9994753003120422, 0.8644062280654907, 0.9320572018623352, 0.9323449730873108, 0.2867054045200348, 0.6933902502059937, 0.3416213393211365, 0.8587242960929871, 0.8462221026420593, 0.5634718537330627, 0.2327478528022766, 0.46145349740982056], "n_positions_probed": 1, "per_restart_best": [10.296488761901855]}
+{"step": 78, "discrete_loss": 11.232571601867676, "best_sample_loss": 10.266305923461914, "soft_loss": 8.84210205078125, "best_discrete": 10.266305923461914, "best_soft": 8.84210205078125, "best_argmax": 11.121016502380371, "best_sampling": 10.266305923461914, "relax_gap": 0.2128158747449208, "n_match": 4, "g_first_norm": 156.91384887695312, "vocab_size": 50257, "entropy": 0.9325403571128845, "entropy_per_token": [1.238431692123413, 1.3081426620483398, 0.5525949597358704, 1.8848150968551636, 0.5062806606292725, 0.672397255897522, 0.011006815358996391, 0.8658356666564941, 0.00514103751629591, 0.5447372198104858, 0.3167244493961334, 0.3917568325996399, 2.3594155311584473, 0.8625420331954956, 1.870896816253662, 0.4271358549594879, 0.7207290530204773, 0.72786945104599, 2.1680548191070557, 1.2162971496582031], "max_p": 0.6969153881072998, "max_p_per_token": [0.6146715879440308, 0.6423508524894714, 0.8708009123802185, 0.3190939426422119, 0.8419402837753296, 0.8041198253631592, 0.9987578392028809, 0.7030209898948669, 0.9995129108428955, 0.868984580039978, 0.936007022857666, 0.9304938912391663, 0.28885141015052795, 0.6842532753944397, 0.3085842430591583, 0.8653208613395691, 0.8355604410171509, 0.5590239763259888, 0.3852015733718872, 0.4817575514316559], "n_positions_probed": 1, "per_restart_best": [10.266305923461914]}
+{"step": 79, "discrete_loss": 11.366512298583984, "best_sample_loss": 10.266305923461914, "soft_loss": 8.996119499206543, "best_discrete": 10.266305923461914, "best_soft": 8.84210205078125, "best_argmax": 11.121016502380371, "best_sampling": 10.266305923461914, "relax_gap": 0.20854178811496468, "n_match": 5, "g_first_norm": 170.18499755859375, "vocab_size": 50257, "entropy": 0.8969488143920898, "entropy_per_token": [1.2592103481292725, 1.3579456806182861, 0.5352901220321655, 1.8657910823822021, 0.5020100474357605, 0.6863417625427246, 0.011277887038886547, 0.8714193105697632, 0.004902651533484459, 0.5408281087875366, 0.30571189522743225, 0.39922624826431274, 2.440661907196045, 0.7939899563789368, 1.8632692098617554, 0.4212428033351898, 0.7461894750595093, 0.7352787256240845, 2.315580368041992, 0.2828086018562317], "max_p": 0.7132354974746704, "max_p_per_token": [0.6045179963111877, 0.6230441331863403, 0.8768507242202759, 0.34878918528556824, 0.8442109227180481, 0.7975738048553467, 0.9987187385559082, 0.6984504461288452, 0.9995391368865967, 0.8708096742630005, 0.9388154745101929, 0.9289534091949463, 0.24505671858787537, 0.7288434505462646, 0.3164113461971283, 0.8692019581794739, 0.8258877396583557, 0.5083215236663818, 0.2992539405822754, 0.9414581656455994], "n_positions_probed": 1, "per_restart_best": [10.266305923461914]}
+{"step": 80, "discrete_loss": 11.392417907714844, "best_sample_loss": 10.266305923461914, "soft_loss": 9.413824081420898, "best_discrete": 10.266305923461914, "best_soft": 8.84210205078125, "best_argmax": 11.121016502380371, "best_sampling": 10.266305923461914, "relax_gap": 0.1736763733846227, "n_match": 6, "g_first_norm": 141.74777221679688, "vocab_size": 50257, "entropy": 0.8423480987548828, "entropy_per_token": [0.05967440828680992, 1.3998913764953613, 0.5412352681159973, 1.8871560096740723, 0.49610960483551025, 0.6730250120162964, 0.011296138167381287, 0.8475808501243591, 0.004643237218260765, 0.533403217792511, 0.29745644330978394, 0.40200167894363403, 2.3898940086364746, 0.7736740112304688, 1.8626725673675537, 0.41178539395332336, 0.778721272945404, 0.7325739860534668, 2.4496254920959473, 0.29454246163368225], "max_p": 0.7307054400444031, "max_p_per_token": [0.991461992263794, 0.6057089567184448, 0.8755993843078613, 0.34921717643737793, 0.8469570875167847, 0.802711546421051, 0.9987118244171143, 0.7101253271102905, 0.9995668530464172, 0.8735195994377136, 0.9409436583518982, 0.9283788800239563, 0.2533305585384369, 0.741176426410675, 0.3026694357395172, 0.874377965927124, 0.8131157755851746, 0.5255207419395447, 0.24311964213848114, 0.937895655632019], "n_positions_probed": 1, "per_restart_best": [10.266305923461914]}
+{"step": 81, "discrete_loss": 11.392417907714844, "best_sample_loss": 10.352619171142578, "soft_loss": 9.494783401489258, "best_discrete": 10.266305923461914, "best_soft": 8.84210205078125, "best_argmax": 11.121016502380371, "best_sampling": 10.266305923461914, "relax_gap": 0.16656995219079215, "n_match": 6, "g_first_norm": 178.07211303710938, "vocab_size": 50257, "entropy": 0.8552476763725281, "entropy_per_token": [0.06276223808526993, 1.5628751516342163, 0.5335958003997803, 1.872245192527771, 0.4982626438140869, 0.6817082166671753, 0.010903866030275822, 0.904564619064331, 0.004258813336491585, 0.505257248878479, 0.28846174478530884, 0.40284082293510437, 2.372222423553467, 0.7830359935760498, 1.876534342765808, 0.4106481075286865, 0.8014348745346069, 0.7289984822273254, 2.5025508403778076, 0.3017934560775757], "max_p": 0.7249178290367126, "max_p_per_token": [0.9909254312515259, 0.552550196647644, 0.8786218762397766, 0.3596855700016022, 0.8457707762718201, 0.7981261014938354, 0.9987590312957764, 0.6712472438812256, 0.9996066689491272, 0.8823524713516235, 0.9431881904602051, 0.9283925890922546, 0.23391731083393097, 0.7373594641685486, 0.3079060912132263, 0.8752590417861938, 0.8044034838676453, 0.5280601382255554, 0.22646141052246094, 0.9357642531394958], "n_positions_probed": 1, "per_restart_best": [10.266305923461914]}
+{"step": 82, "discrete_loss": 11.46057415008545, "best_sample_loss": 10.266305923461914, "soft_loss": 9.36966609954834, "best_discrete": 10.266305923461914, "best_soft": 8.84210205078125, "best_argmax": 11.121016502380371, "best_sampling": 10.266305923461914, "relax_gap": 0.1824435689831054, "n_match": 7, "g_first_norm": 138.64556884765625, "vocab_size": 50257, "entropy": 0.8487168550491333, "entropy_per_token": [0.0654851496219635, 1.5450955629348755, 0.3114015758037567, 1.8654804229736328, 0.5009068250656128, 0.6993521451950073, 0.0108176926150918, 0.9735118746757507, 0.003929235972464085, 0.483814001083374, 0.27776023745536804, 0.4062255620956421, 2.3649673461914062, 0.7783763408660889, 1.872762680053711, 0.40818867087364197, 0.8230501413345337, 0.7264924049377441, 2.5393404960632324, 0.3173774182796478], "max_p": 0.7232298254966736, "max_p_per_token": [0.9904468059539795, 0.5584800839424133, 0.9176032543182373, 0.36628398299217224, 0.8442339897155762, 0.7894010543823242, 0.9987665414810181, 0.6175659894943237, 0.9996404647827148, 0.8891481757164001, 0.9458823800086975, 0.9277826547622681, 0.2304888665676117, 0.740927517414093, 0.3166765868663788, 0.8769952654838562, 0.7953484058380127, 0.5303356647491455, 0.19761475920677185, 0.9309735298156738], "n_positions_probed": 1, "per_restart_best": [10.266305923461914]}
+{"step": 83, "discrete_loss": 11.46057415008545, "best_sample_loss": 10.359467506408691, "soft_loss": 9.335805892944336, "best_discrete": 10.266305923461914, "best_soft": 8.84210205078125, "best_argmax": 11.121016502380371, "best_sampling": 10.266305923461914, "relax_gap": 0.18539806377198573, "n_match": 7, "g_first_norm": 127.79698181152344, "vocab_size": 50257, "entropy": 0.8664854168891907, "entropy_per_token": [0.06836030632257462, 1.5321550369262695, 0.3157939910888672, 2.1030592918395996, 0.5049132108688354, 0.7229598164558411, 0.010969465598464012, 1.0255167484283447, 0.003661290742456913, 0.475521445274353, 0.27286025881767273, 0.41645103693008423, 2.348734140396118, 0.7827283143997192, 1.8641536235809326, 0.40397337079048157, 0.8485385179519653, 0.7243378758430481, 2.565178155899048, 0.3398413360118866], "max_p": 0.7151392102241516, "max_p_per_token": [0.9899178147315979, 0.5621907114982605, 0.9160774946212769, 0.2849480211734772, 0.841688871383667, 0.7779802680015564, 0.998742401599884, 0.5747793912887573, 0.9996678829193115, 0.891880214214325, 0.9471802115440369, 0.9255456328392029, 0.22422969341278076, 0.7390227317810059, 0.3087189495563507, 0.8796908855438232, 0.7835661768913269, 0.5349335074424744, 0.19821490347385406, 0.9238079786300659], "n_positions_probed": 1, "per_restart_best": [10.266305923461914]}
+{"step": 84, "discrete_loss": 11.46057415008545, "best_sample_loss": 10.273364067077637, "soft_loss": 9.145790100097656, "best_discrete": 10.266305923461914, "best_soft": 8.84210205078125, "best_argmax": 11.121016502380371, "best_sampling": 10.266305923461914, "relax_gap": 0.20197801782649205, "n_match": 7, "g_first_norm": 131.7224578857422, "vocab_size": 50257, "entropy": 0.8724063038825989, "entropy_per_token": [0.07214416563510895, 1.5277516841888428, 0.321386456489563, 2.0592715740203857, 0.5133182406425476, 0.7390683889389038, 0.011051801964640617, 1.0904170274734497, 0.003372696228325367, 0.4638206660747528, 0.2663072645664215, 0.427863210439682, 2.3493876457214355, 0.7925246357917786, 1.865417718887329, 0.4040243327617645, 0.8633949756622314, 0.7225363254547119, 2.585939645767212, 0.36912718415260315], "max_p": 0.710861325263977, "max_p_per_token": [0.9892113208770752, 0.5613430142402649, 0.9140738844871521, 0.3131483793258667, 0.8393787145614624, 0.770281195640564, 0.9987275004386902, 0.5098217129707336, 0.9996970891952515, 0.8956440091133118, 0.9488548040390015, 0.9229910373687744, 0.22213084995746613, 0.7342541813850403, 0.30808016657829285, 0.8803392052650452, 0.7735611796379089, 0.535336971282959, 0.18617568910121918, 0.9141747355461121], "n_positions_probed": 1, "per_restart_best": [10.266305923461914]}
+{"step": 85, "discrete_loss": 11.46057415008545, "best_sample_loss": 10.493288040161133, "soft_loss": 9.066827774047852, "best_discrete": 10.266305923461914, "best_soft": 8.84210205078125, "best_argmax": 11.121016502380371, "best_sampling": 10.266305923461914, "relax_gap": 0.2088679279667459, "n_match": 7, "g_first_norm": 127.52212524414062, "vocab_size": 50257, "entropy": 0.8792405128479004, "entropy_per_token": [0.07527919113636017, 1.5280917882919312, 0.3249850571155548, 2.0637307167053223, 0.5189297199249268, 0.7980203628540039, 0.011151342652738094, 1.1040886640548706, 0.003110234858468175, 0.4492988586425781, 0.2596087157726288, 0.4406169652938843, 2.345862865447998, 0.8061489462852478, 1.8698930740356445, 0.4023781418800354, 0.875607967376709, 0.720573902130127, 2.5861687660217285, 0.4012645483016968], "max_p": 0.7063994407653809, "max_p_per_token": [0.9886088371276855, 0.5578969120979309, 0.912760853767395, 0.31015485525131226, 0.8362836241722107, 0.7536906599998474, 0.9987102746963501, 0.47864753007888794, 0.9997233748435974, 0.9001474976539612, 0.9505330324172974, 0.9200974702835083, 0.2225867211818695, 0.7274748086929321, 0.3083617091178894, 0.8818235993385315, 0.7633006572723389, 0.5344181656837463, 0.1796613186597824, 0.9031066298484802], "n_positions_probed": 1, "per_restart_best": [10.266305923461914]}
+{"step": 86, "discrete_loss": 11.46057415008545, "best_sample_loss": 10.378700256347656, "soft_loss": 9.007412910461426, "best_discrete": 10.266305923461914, "best_soft": 8.84210205078125, "best_argmax": 11.121016502380371, "best_sampling": 10.266305923461914, "relax_gap": 0.21405221130267132, "n_match": 7, "g_first_norm": 125.9971694946289, "vocab_size": 50257, "entropy": 0.8840678334236145, "entropy_per_token": [0.078712098300457, 1.5282448530197144, 0.3298894762992859, 2.0609524250030518, 0.5237573981285095, 0.8360500335693359, 0.013328303582966328, 1.1049145460128784, 0.002886928152292967, 0.4356352686882019, 0.2529275715351105, 0.45489346981048584, 2.343989849090576, 0.8211806416511536, 1.8747820854187012, 0.3998781442642212, 0.8862276077270508, 0.7189042568206787, 2.5768141746520996, 0.4373868703842163], "max_p": 0.7023550271987915, "max_p_per_token": [0.987941324710846, 0.5546761751174927, 0.9109196662902832, 0.31099173426628113, 0.8335199356079102, 0.7351636290550232, 0.9984514713287354, 0.4593627154827118, 0.9997454285621643, 0.9043410420417786, 0.9521815180778503, 0.9167888760566711, 0.2264690101146698, 0.719902753829956, 0.30626508593559265, 0.8837460875511169, 0.75223308801651, 0.5327929258346558, 0.1715654730796814, 0.8900417685508728], "n_positions_probed": 1, "per_restart_best": [10.266305923461914]}
+{"step": 87, "discrete_loss": 11.419516563415527, "best_sample_loss": 10.404825210571289, "soft_loss": 8.952807426452637, "best_discrete": 10.266305923461914, "best_soft": 8.84210205078125, "best_argmax": 11.121016502380371, "best_sampling": 10.266305923461914, "relax_gap": 0.21600819292696125, "n_match": 7, "g_first_norm": 126.45906829833984, "vocab_size": 50257, "entropy": 0.8829509615898132, "entropy_per_token": [0.08202338963747025, 1.5266458988189697, 0.3353072702884674, 2.0575873851776123, 0.5280392169952393, 0.8699723482131958, 0.013415702618658543, 1.0024919509887695, 0.002684582956135273, 0.42232370376586914, 0.2466762661933899, 0.4712659418582916, 2.3444113731384277, 0.8365037441253662, 1.8800034523010254, 0.39690980315208435, 0.893596887588501, 0.7172592878341675, 2.553831100463867, 0.47807031869888306], "max_p": 0.7104020714759827, "max_p_per_token": [0.9872822761535645, 0.5523619651794434, 0.9088502526283264, 0.31183016300201416, 0.8309894800186157, 0.7177696824073792, 0.9984351992607117, 0.6802445650100708, 0.9997654557228088, 0.9083631038665771, 0.9537106156349182, 0.9129217863082886, 0.23116523027420044, 0.711990475654602, 0.30511561036109924, 0.8859238028526306, 0.7408698201179504, 0.5313234329223633, 0.16456493735313416, 0.8745637536048889], "n_positions_probed": 1, "per_restart_best": [10.266305923461914]}
+{"step": 88, "discrete_loss": 11.419516563415527, "best_sample_loss": 10.242866516113281, "soft_loss": 9.05850601196289, "best_discrete": 10.242866516113281, "best_soft": 8.84210205078125, "best_argmax": 11.121016502380371, "best_sampling": 10.242866516113281, "relax_gap": 0.20675223319142583, "n_match": 7, "g_first_norm": 129.4613800048828, "vocab_size": 50257, "entropy": 0.8825253844261169, "entropy_per_token": [0.08624732494354248, 1.5097414255142212, 0.33512401580810547, 2.0480713844299316, 0.5323469638824463, 0.9092763662338257, 0.013459491543471813, 0.9744898080825806, 0.005662280600517988, 0.4168257713317871, 0.24018828570842743, 0.4779873490333557, 2.318230152130127, 0.8418048620223999, 1.8733484745025635, 0.3956837058067322, 0.8870024085044861, 0.7166870832443237, 2.5521743297576904, 0.5161545276641846], "max_p": 0.7102729678153992, "max_p_per_token": [0.9864808320999146, 0.558931827545166, 0.9088866114616394, 0.31720057129859924, 0.8281806707382202, 0.6980084180831909, 0.9984270334243774, 0.6953579783439636, 0.9994547963142395, 0.9099960923194885, 0.9553180932998657, 0.9112457633018494, 0.2532361149787903, 0.7102125883102417, 0.3136310577392578, 0.8873955011367798, 0.733736515045166, 0.5273846983909607, 0.1530311405658722, 0.8593428134918213], "n_positions_probed": 1, "per_restart_best": [10.242866516113281]}
+{"step": 89, "discrete_loss": 11.27042293548584, "best_sample_loss": 10.265765190124512, "soft_loss": 9.007533073425293, "best_discrete": 10.242866516113281, "best_soft": 8.84210205078125, "best_argmax": 11.121016502380371, "best_sampling": 10.242866516113281, "relax_gap": 0.2007812728070439, "n_match": 7, "g_first_norm": 135.5555419921875, "vocab_size": 50257, "entropy": 0.8798196911811829, "entropy_per_token": [0.09564392268657684, 1.4889965057373047, 0.33359426259994507, 2.0486950874328613, 0.5363186001777649, 0.9937838315963745, 0.013226844370365143, 0.9496333599090576, 0.005269207060337067, 0.31275659799575806, 0.23429694771766663, 0.4951810836791992, 2.298492670059204, 0.8499613404273987, 1.8738901615142822, 0.39566805958747864, 0.8767020106315613, 0.7153065800666809, 2.5091869831085205, 0.5697895884513855], "max_p": 0.7102965712547302, "max_p_per_token": [0.9846549034118652, 0.5665034651756287, 0.9093899130821228, 0.3151327073574066, 0.8256595134735107, 0.6524596214294434, 0.998456597328186, 0.7082570195198059, 0.999496579170227, 0.9404982924461365, 0.9567384123802185, 0.907052755355835, 0.2670901119709015, 0.7062981724739075, 0.3200482130050659, 0.888392984867096, 0.7273756265640259, 0.5196241140365601, 0.1757126748561859, 0.8370901942253113], "n_positions_probed": 1, "per_restart_best": [10.242866516113281]}
+{"step": 90, "discrete_loss": 11.27042293548584, "best_sample_loss": 10.28330135345459, "soft_loss": 8.940372467041016, "best_discrete": 10.242866516113281, "best_soft": 8.84210205078125, "best_argmax": 11.121016502380371, "best_sampling": 10.242866516113281, "relax_gap": 0.20674028665849545, "n_match": 7, "g_first_norm": 134.368896484375, "vocab_size": 50257, "entropy": 0.8885158896446228, "entropy_per_token": [0.10333241522312164, 1.4769847393035889, 0.334734171628952, 2.0382165908813477, 0.5392951369285583, 1.0559886693954468, 0.013087262399494648, 0.9378702640533447, 0.004930097609758377, 0.30822446942329407, 0.2352522909641266, 0.515143632888794, 2.2861199378967285, 0.8582413792610168, 1.8791797161102295, 0.3955671191215515, 0.8699131608009338, 0.7137689590454102, 2.510983943939209, 0.6934829950332642], "max_p": 0.7056809067726135, "max_p_per_token": [0.983112633228302, 0.5706092119216919, 0.9088667035102844, 0.32013460993766785, 0.8237903714179993, 0.6155601739883423, 0.9984732270240784, 0.7146445512771606, 0.9995322227478027, 0.9414622187614441, 0.957250714302063, 0.9021530747413635, 0.2780545651912689, 0.7018634080886841, 0.31613293290138245, 0.8893490433692932, 0.7173332571983337, 0.5236498713493347, 0.17119397222995758, 0.780450701713562], "n_positions_probed": 1, "per_restart_best": [10.242866516113281]}
+{"step": 91, "discrete_loss": 11.27042293548584, "best_sample_loss": 10.327349662780762, "soft_loss": 8.833246231079102, "best_discrete": 10.242866516113281, "best_soft": 8.833246231079102, "best_argmax": 11.121016502380371, "best_sampling": 10.242866516113281, "relax_gap": 0.21624536349324477, "n_match": 7, "g_first_norm": 145.6126708984375, "vocab_size": 50257, "entropy": 0.9002147912979126, "entropy_per_token": [0.11178061366081238, 1.4625588655471802, 0.334093302488327, 2.041193962097168, 0.5447105169296265, 1.035491704940796, 0.012677688151597977, 0.9363454580307007, 0.0045542302541434765, 0.30252784490585327, 0.22812984883785248, 0.567466139793396, 2.244858980178833, 0.9078546762466431, 1.904879093170166, 0.39377978444099426, 0.8675898909568787, 0.7121655941009521, 2.5005292892456055, 0.8911083340644836], "max_p": 0.6989123225212097, "max_p_per_token": [0.9813398718833923, 0.5759220123291016, 0.908988356590271, 0.3167734146118164, 0.8208823800086975, 0.6266981363296509, 0.9985264539718628, 0.716387927532196, 0.9995712637901306, 0.9426464438438416, 0.958884596824646, 0.8916193246841431, 0.31286031007766724, 0.6703845858573914, 0.30549290776252747, 0.8910358548164368, 0.7017637491226196, 0.5205434560775757, 0.16964712738990784, 0.66827791929245], "n_positions_probed": 1, "per_restart_best": [10.242866516113281]}
+{"step": 92, "discrete_loss": 11.045506477355957, "best_sample_loss": 10.275301933288574, "soft_loss": 8.69027328491211, "best_discrete": 10.242866516113281, "best_soft": 8.69027328491211, "best_argmax": 11.045506477355957, "best_sampling": 10.242866516113281, "relax_gap": 0.21322998608277824, "n_match": 7, "g_first_norm": 136.84120178222656, "vocab_size": 50257, "entropy": 0.8632190823554993, "entropy_per_token": [0.1203107237815857, 1.444716453552246, 0.33140161633491516, 2.033947467803955, 0.5486564636230469, 1.0368144512176514, 0.012294666841626167, 0.9428697228431702, 0.004304237198084593, 0.30003997683525085, 0.22003960609436035, 0.590392529964447, 1.339883804321289, 0.9620412588119507, 1.9349571466445923, 0.3942026197910309, 0.8633280992507935, 0.7110370397567749, 2.5097298622131348, 0.9634122252464294], "max_p": 0.6992368698120117, "max_p_per_token": [0.9794843792915344, 0.5823726654052734, 0.9099261164665222, 0.3206334412097931, 0.8187843561172485, 0.6247278451919556, 0.9985755681991577, 0.7143375277519226, 0.9995971322059631, 0.9429892301559448, 0.9607316255569458, 0.8858916163444519, 0.44994011521339417, 0.6313509345054626, 0.2998591661453247, 0.8919881582260132, 0.6849871277809143, 0.5145013928413391, 0.1635773628950119, 0.6104817986488342], "n_positions_probed": 1, "per_restart_best": [10.242866516113281]}
+{"step": 93, "discrete_loss": 11.27042293548584, "best_sample_loss": 10.386770248413086, "soft_loss": 9.027544021606445, "best_discrete": 10.242866516113281, "best_soft": 8.69027328491211, "best_argmax": 11.045506477355957, "best_sampling": 10.242866516113281, "relax_gap": 0.19900574510096763, "n_match": 7, "g_first_norm": 181.25128173828125, "vocab_size": 50257, "entropy": 0.8632608652114868, "entropy_per_token": [0.12573477625846863, 1.4236345291137695, 0.3302726149559021, 2.057116746902466, 0.5530994534492493, 0.9656351804733276, 0.011956913396716118, 0.9340771436691284, 0.004058374557644129, 0.29810282588005066, 0.21867050230503082, 0.6295610666275024, 1.4444241523742676, 0.9901986122131348, 1.9356595277786255, 0.40536952018737793, 0.8402153849601746, 0.7075258493423462, 2.4262514114379883, 0.9636516571044922], "max_p": 0.7048279047012329, "max_p_per_token": [0.978342592716217, 0.5889822840690613, 0.9103887677192688, 0.3053906559944153, 0.8167660236358643, 0.6668953895568848, 0.998621940612793, 0.7202563285827637, 0.9996222257614136, 0.9430616497993469, 0.9612246155738831, 0.8758076429367065, 0.45803841948509216, 0.6298858523368835, 0.34935620427131653, 0.8886111974716187, 0.6774027943611145, 0.5281389951705933, 0.21183718740940094, 0.5879266858100891], "n_positions_probed": 1, "per_restart_best": [10.242866516113281]}
+{"step": 94, "discrete_loss": 11.612471580505371, "best_sample_loss": 10.092340469360352, "soft_loss": 8.921099662780762, "best_discrete": 10.092340469360352, "best_soft": 8.69027328491211, "best_argmax": 11.045506477355957, "best_sampling": 10.092340469360352, "relax_gap": 0.23176564085140966, "n_match": 7, "g_first_norm": 147.90420532226562, "vocab_size": 50257, "entropy": 0.828207790851593, "entropy_per_token": [0.12962204217910767, 1.4103634357452393, 0.3325767517089844, 2.0348384380340576, 0.5587527751922607, 1.0297495126724243, 0.011870148591697216, 0.9516608715057373, 0.003970946650952101, 0.3009394705295563, 0.2114362269639969, 0.6460726261138916, 1.557267665863037, 1.0232630968093872, 0.863911509513855, 0.41509270668029785, 0.8522793650627136, 0.7056374549865723, 2.50642728805542, 1.0184245109558105], "max_p": 0.7164668440818787, "max_p_per_token": [0.9775974154472351, 0.5936744213104248, 0.9095483422279358, 0.31839820742607117, 0.8129972219467163, 0.6292117238044739, 0.9986332058906555, 0.7129288911819458, 0.9996304512023926, 0.9421855807304382, 0.9628485441207886, 0.8711928725242615, 0.47213077545166016, 0.6048834919929504, 0.8032339215278625, 0.8856324553489685, 0.6450053453445435, 0.5358010530471802, 0.17823180556297302, 0.4755706191062927], "n_positions_probed": 1, "per_restart_best": [10.092340469360352]}
+{"step": 95, "discrete_loss": 11.42088794708252, "best_sample_loss": 10.135161399841309, "soft_loss": 10.039766311645508, "best_discrete": 10.092340469360352, "best_soft": 8.69027328491211, "best_argmax": 11.045506477355957, "best_sampling": 10.092340469360352, "relax_gap": 0.12092944452623064, "n_match": 7, "g_first_norm": 216.26504516601562, "vocab_size": 50257, "entropy": 0.8674732446670532, "entropy_per_token": [0.13709940016269684, 1.3567626476287842, 0.3313286602497101, 2.1019091606140137, 0.5774667859077454, 0.9481088519096375, 0.011850222945213318, 0.9848947525024414, 0.003971894271671772, 0.31151455640792847, 0.20564596354961395, 0.7014299035072327, 1.8523285388946533, 1.0556625127792358, 0.9703313708305359, 0.7984326481819153, 0.8390599489212036, 0.6958528757095337, 2.428152561187744, 1.0376611948013306], "max_p": 0.7087330222129822, "max_p_per_token": [0.9760231971740723, 0.6153562068939209, 0.9099062085151672, 0.269669771194458, 0.8016902208328247, 0.6774953007698059, 0.998637855052948, 0.6982226967811584, 0.9996300935745239, 0.9393290877342224, 0.964197039604187, 0.8562163710594177, 0.40162500739097595, 0.5945123434066772, 0.7686732411384583, 0.7788538932800293, 0.6237483024597168, 0.5710766315460205, 0.20805102586746216, 0.5217454433441162], "n_positions_probed": 1, "per_restart_best": [10.092340469360352]}
+{"step": 96, "discrete_loss": 11.50864315032959, "best_sample_loss": 10.241058349609375, "soft_loss": 9.684867858886719, "best_discrete": 10.092340469360352, "best_soft": 8.69027328491211, "best_argmax": 11.045506477355957, "best_sampling": 10.092340469360352, "relax_gap": 0.15847005312617074, "n_match": 6, "g_first_norm": 198.25177001953125, "vocab_size": 50257, "entropy": 0.9081351161003113, "entropy_per_token": [0.14378251135349274, 1.3421247005462646, 0.3302657902240753, 2.0635986328125, 0.5827871561050415, 1.0639402866363525, 0.01209633145481348, 0.9985845685005188, 0.003899992909282446, 0.3097667098045349, 0.19748039543628693, 0.7368612289428711, 2.0452277660369873, 1.0581918954849243, 1.129103660583496, 0.8373603820800781, 1.007802963256836, 0.6885885000228882, 2.5341293811798096, 1.0771093368530273], "max_p": 0.6923040747642517, "max_p_per_token": [0.9746429920196533, 0.6198374629020691, 0.910305380821228, 0.29049500823020935, 0.7979940176010132, 0.6086001396179199, 0.9986047148704529, 0.6942002773284912, 0.9996370077133179, 0.9395537972450256, 0.9660245180130005, 0.8464205265045166, 0.3727300465106964, 0.5791293978691101, 0.7100281119346619, 0.7626043558120728, 0.5421433448791504, 0.591484546661377, 0.14370860159397125, 0.49793681502342224], "n_positions_probed": 1, "per_restart_best": [10.092340469360352]}
+{"step": 97, "discrete_loss": 11.494375228881836, "best_sample_loss": 10.254439353942871, "soft_loss": 9.463037490844727, "best_discrete": 10.092340469360352, "best_soft": 8.69027328491211, "best_argmax": 11.045506477355957, "best_sampling": 10.092340469360352, "relax_gap": 0.17672450199232936, "n_match": 7, "g_first_norm": 236.37991333007812, "vocab_size": 50257, "entropy": 0.9126785397529602, "entropy_per_token": [0.15029078722000122, 1.2962931394577026, 0.32535117864608765, 2.0563015937805176, 0.5954515933990479, 0.9631505608558655, 0.011908095329999924, 1.0161669254302979, 0.003744086716324091, 0.31086111068725586, 0.1877737045288086, 0.8087760806083679, 2.3029704093933105, 1.1053816080093384, 1.6603517532348633, 0.8744105100631714, 0.9573076963424683, 0.09878341853618622, 2.4165754318237305, 1.1117198467254639], "max_p": 0.69683438539505, "max_p_per_token": [0.9732441306114197, 0.636339008808136, 0.9121594429016113, 0.28833508491516113, 0.7903536558151245, 0.6692798137664795, 0.998630166053772, 0.6884503364562988, 0.9996529817581177, 0.9388835430145264, 0.968148946762085, 0.8259029388427734, 0.28201642632484436, 0.5695005655288696, 0.4557330012321472, 0.7468064427375793, 0.49044495820999146, 0.9799228310585022, 0.22306250035762787, 0.49981993436813354], "n_positions_probed": 1, "per_restart_best": [10.092340469360352]}
+{"step": 98, "discrete_loss": 10.99799633026123, "best_sample_loss": 10.144085884094238, "soft_loss": 8.966501235961914, "best_discrete": 10.092340469360352, "best_soft": 8.69027328491211, "best_argmax": 10.99799633026123, "best_sampling": 10.092340469360352, "relax_gap": 0.1847150183810857, "n_match": 8, "g_first_norm": 341.57452392578125, "vocab_size": 50257, "entropy": 0.9095927476882935, "entropy_per_token": [0.1661657840013504, 1.282335877418518, 0.3127867579460144, 2.0685298442840576, 0.5987175703048706, 1.1201274394989014, 0.01252642460167408, 0.9776456356048584, 0.0034084836952388287, 0.3010368347167969, 0.18165118992328644, 0.8379030823707581, 2.298628330230713, 0.982200562953949, 1.5431485176086426, 0.9002392888069153, 0.930367112159729, 0.10134172439575195, 2.4224741458892822, 1.1506195068359375], "max_p": 0.6975197196006775, "max_p_per_token": [0.9695234894752502, 0.6352249979972839, 0.9168775677680969, 0.2704567611217499, 0.7880780696868896, 0.5715962052345276, 0.9985460042953491, 0.7112759947776794, 0.9996867179870605, 0.9409314393997192, 0.9696041941642761, 0.8174579739570618, 0.31544771790504456, 0.6339581608772278, 0.5351790189743042, 0.7357495427131653, 0.5027927160263062, 0.9792646169662476, 0.18365047872066498, 0.4750916659832001], "n_positions_probed": 1, "per_restart_best": [10.092340469360352]}
+{"step": 99, "discrete_loss": 10.998148918151855, "best_sample_loss": 10.064926147460938, "soft_loss": 8.578283309936523, "best_discrete": 10.064926147460938, "best_soft": 8.578283309936523, "best_argmax": 10.99799633026123, "best_sampling": 10.064926147460938, "relax_gap": 0.2200248083767509, "n_match": 6, "g_first_norm": 209.45578002929688, "vocab_size": 50257, "entropy": 0.8858124017715454, "entropy_per_token": [0.1882992386817932, 1.2770090103149414, 0.316389799118042, 2.0520379543304443, 0.6235819458961487, 1.0259668827056885, 0.012220825999975204, 1.0095622539520264, 0.0033130417577922344, 0.2957562506198883, 0.17809002101421356, 0.866037130355835, 2.225984811782837, 1.1038769483566284, 1.779468297958374, 0.9106500148773193, 0.894313395023346, 0.10021867603063583, 2.441119432449341, 0.41235119104385376], "max_p": 0.7131596803665161, "max_p_per_token": [0.9640223383903503, 0.6327297687530518, 0.9155716300010681, 0.27752378582954407, 0.7727620005607605, 0.6308974623680115, 0.998586893081665, 0.6990519165992737, 0.9996961355209351, 0.9419751167297363, 0.9704121947288513, 0.8087778091430664, 0.3565848469734192, 0.5391861796379089, 0.34609219431877136, 0.7290166020393372, 0.5788105130195618, 0.9795539379119873, 0.2145046442747116, 0.9074372053146362], "n_positions_probed": 1, "per_restart_best": [10.064926147460938]}
+{"step": 100, "discrete_loss": 10.998148918151855, "best_sample_loss": 10.165380477905273, "soft_loss": 8.854330062866211, "best_discrete": 10.064926147460938, "best_soft": 8.578283309936523, "best_argmax": 10.99799633026123, "best_sampling": 10.064926147460938, "relax_gap": 0.19492542529110388, "n_match": 6, "g_first_norm": 166.03379821777344, "vocab_size": 50257, "entropy": 0.8833967447280884, "entropy_per_token": [0.07533501088619232, 1.2678014039993286, 0.3161204159259796, 2.012740135192871, 0.6249365210533142, 1.0009390115737915, 0.012200583703815937, 0.9926491975784302, 0.00316803902387619, 0.29389840364456177, 0.17261889576911926, 0.8758573532104492, 2.4061203002929688, 1.1184039115905762, 1.7658448219299316, 0.9258368015289307, 0.8358121514320374, 0.10412900149822235, 2.4564530849456787, 0.4070703685283661], "max_p": 0.7178115844726562, "max_p_per_token": [0.9894455075263977, 0.6340510249137878, 0.9156902432441711, 0.30435463786125183, 0.7709425091743469, 0.6455273628234863, 0.9985871315002441, 0.7074598670005798, 0.9997100234031677, 0.9422546029090881, 0.9715675115585327, 0.8058511018753052, 0.23742610216140747, 0.5594528913497925, 0.36860984563827515, 0.7198695540428162, 0.6527971029281616, 0.9785423278808594, 0.2452109009027481, 0.9088810682296753], "n_positions_probed": 1, "per_restart_best": [10.064926147460938]}
+{"step": 101, "discrete_loss": 10.998148918151855, "best_sample_loss": 10.126524925231934, "soft_loss": 8.73619270324707, "best_discrete": 10.064926147460938, "best_soft": 8.578283309936523, "best_argmax": 10.99799633026123, "best_sampling": 10.064926147460938, "relax_gap": 0.20566699284927373, "n_match": 6, "g_first_norm": 160.85780334472656, "vocab_size": 50257, "entropy": 0.8933143615722656, "entropy_per_token": [0.08758865296840668, 1.417047142982483, 0.3164919316768646, 2.0373575687408447, 0.633709192276001, 1.0529141426086426, 0.012208763509988785, 0.9926466941833496, 0.00303982337936759, 0.2946220636367798, 0.1668458729982376, 0.8501814603805542, 2.2686119079589844, 1.159401774406433, 1.7616045475006104, 0.9461174011230469, 0.8012975454330444, 0.10568130016326904, 2.5358262062072754, 0.42309319972991943], "max_p": 0.706802487373352, "max_p_per_token": [0.9874211549758911, 0.4443480670452118, 0.9154515862464905, 0.2887864112854004, 0.7640370726585388, 0.6118259429931641, 0.998584508895874, 0.7078569531440735, 0.9997226595878601, 0.9420048594474792, 0.9727354645729065, 0.8120602369308472, 0.30230289697647095, 0.5413081645965576, 0.3600492775440216, 0.7065637111663818, 0.691302478313446, 0.9781369566917419, 0.20807407796382904, 0.9034761786460876], "n_positions_probed": 1, "per_restart_best": [10.064926147460938]}
+{"step": 102, "discrete_loss": 10.998148918151855, "best_sample_loss": 10.066696166992188, "soft_loss": 8.637334823608398, "best_discrete": 10.064926147460938, "best_soft": 8.578283309936523, "best_argmax": 10.99799633026123, "best_sampling": 10.064926147460938, "relax_gap": 0.21465558541829344, "n_match": 6, "g_first_norm": 155.609619140625, "vocab_size": 50257, "entropy": 0.891876220703125, "entropy_per_token": [0.08849973231554031, 1.415350079536438, 0.31544142961502075, 2.0378003120422363, 0.6492248773574829, 0.9856595993041992, 0.012294750660657883, 0.9792795181274414, 0.0028607856947928667, 0.28967803716659546, 0.1618005484342575, 0.8540757298469543, 2.3041739463806152, 1.1785805225372314, 1.758367896080017, 0.9663430452346802, 0.7528694868087769, 0.11036829650402069, 2.5446462631225586, 0.4302091598510742], "max_p": 0.7099121809005737, "max_p_per_token": [0.9872918128967285, 0.4496283531188965, 0.9166499972343445, 0.2871386408805847, 0.753291130065918, 0.6520301699638367, 0.9985710382461548, 0.714911937713623, 0.9997404217720032, 0.9430293440818787, 0.9737902283668518, 0.8102091550827026, 0.2713174819946289, 0.5234461426734924, 0.3852279484272003, 0.6906903982162476, 0.7296652793884277, 0.9769008755683899, 0.2333272099494934, 0.9013853073120117], "n_positions_probed": 1, "per_restart_best": [10.064926147460938]}
+{"step": 103, "discrete_loss": 10.998148918151855, "best_sample_loss": 10.199568748474121, "soft_loss": 8.578136444091797, "best_discrete": 10.064926147460938, "best_soft": 8.578136444091797, "best_argmax": 10.99799633026123, "best_sampling": 10.064926147460938, "relax_gap": 0.22003816206434137, "n_match": 6, "g_first_norm": 152.6456756591797, "vocab_size": 50257, "entropy": 0.9020295143127441, "entropy_per_token": [0.09933900088071823, 1.4048362970352173, 0.3104473948478699, 2.042942523956299, 0.6632825136184692, 1.0774188041687012, 0.012351501733064651, 0.9793514013290405, 0.0027174088172614574, 0.2880396246910095, 0.15593089163303375, 0.8482885956764221, 2.247859001159668, 1.22183358669281, 1.760613203048706, 0.9896199703216553, 0.7304327487945557, 0.11358091980218887, 2.641496419906616, 0.4502084255218506], "max_p": 0.7030437588691711, "max_p_per_token": [0.9854705333709717, 0.4580115079879761, 0.9184075593948364, 0.28769728541374207, 0.7428339719772339, 0.5906944870948792, 0.9985615611076355, 0.7156520485877991, 0.9997543692588806, 0.9433035850524902, 0.9749593734741211, 0.8107722401618958, 0.30219435691833496, 0.48323145508766174, 0.3774258494377136, 0.6713250279426575, 0.7478243708610535, 0.9760427474975586, 0.18176129460334778, 0.8949510455131531], "n_positions_probed": 1, "per_restart_best": [10.064926147460938]}
+{"step": 104, "discrete_loss": 10.998148918151855, "best_sample_loss": 10.14389705657959, "soft_loss": 8.517245292663574, "best_discrete": 10.064926147460938, "best_soft": 8.517245292663574, "best_argmax": 10.99799633026123, "best_sampling": 10.064926147460938, "relax_gap": 0.22557465296670812, "n_match": 6, "g_first_norm": 171.8431396484375, "vocab_size": 50257, "entropy": 0.8984332084655762, "entropy_per_token": [0.09908133745193481, 1.4053360223770142, 0.3061218857765198, 2.049398422241211, 0.7230904698371887, 0.9786081314086914, 0.01260833814740181, 0.962499737739563, 0.0025223479606211185, 0.28221991658210754, 0.15294018387794495, 0.8621514439582825, 2.297312021255493, 1.2223103046417236, 1.7315797805786133, 1.0024420022964478, 0.6963317394256592, 0.11951176077127457, 2.6144747734069824, 0.4481245279312134], "max_p": 0.7064402103424072, "max_p_per_token": [0.9855316877365112, 0.46140819787979126, 0.9199557304382324, 0.2810099720954895, 0.7207978367805481, 0.6522098183631897, 0.9985255599021912, 0.7245513200759888, 0.9997735619544983, 0.944550096988678, 0.9756185412406921, 0.806107223033905, 0.25748205184936523, 0.4561641216278076, 0.425521582365036, 0.653910219669342, 0.7692096829414368, 0.9744375944137573, 0.22612978518009186, 0.89590984582901], "n_positions_probed": 1, "per_restart_best": [10.064926147460938]}
+{"step": 105, "discrete_loss": 10.998148918151855, "best_sample_loss": 10.283125877380371, "soft_loss": 8.452445983886719, "best_discrete": 10.064926147460938, "best_soft": 8.452445983886719, "best_argmax": 10.99799633026123, "best_sampling": 10.064926147460938, "relax_gap": 0.23146649069859296, "n_match": 6, "g_first_norm": 156.65625, "vocab_size": 50257, "entropy": 0.9095972180366516, "entropy_per_token": [0.11528681218624115, 1.3918251991271973, 0.30090707540512085, 2.043447971343994, 0.7411643266677856, 1.0874289274215698, 0.012712525203824043, 0.9680607318878174, 0.0023858651984483004, 0.2812722325325012, 0.14825774729251862, 0.8566927909851074, 2.221776247024536, 1.2651088237762451, 1.734190821647644, 1.0236163139343262, 0.6881754994392395, 0.12234549224376678, 2.716414451599121, 0.47087353467941284], "max_p": 0.6972846984863281, "max_p_per_token": [0.9827234745025635, 0.4687727689743042, 0.9217565655708313, 0.28557562828063965, 0.7076074481010437, 0.5905613303184509, 0.9985095858573914, 0.7230663895606995, 0.999786913394928, 0.9446583986282349, 0.9765347242355347, 0.8064723610877991, 0.3051625192165375, 0.39317429065704346, 0.41888627409935, 0.6302492022514343, 0.7761086225509644, 0.9736595749855042, 0.1539388746023178, 0.8884890675544739], "n_positions_probed": 1, "per_restart_best": [10.064926147460938]}
+{"step": 106, "discrete_loss": 10.998148918151855, "best_sample_loss": 10.207892417907715, "soft_loss": 8.370854377746582, "best_discrete": 10.064926147460938, "best_soft": 8.370854377746582, "best_argmax": 10.99799633026123, "best_sampling": 10.064926147460938, "relax_gap": 0.23888515785316058, "n_match": 6, "g_first_norm": 171.96044921875, "vocab_size": 50257, "entropy": 0.9020944833755493, "entropy_per_token": [0.11904145777225494, 1.3882453441619873, 0.2961967885494232, 2.0447704792022705, 0.7707593441009521, 0.9792560338973999, 0.01331685483455658, 0.954641580581665, 0.0022194632329046726, 0.2756481468677521, 0.14540192484855652, 0.8745971918106079, 2.262014865875244, 1.2759809494018555, 1.6870248317718506, 1.0247284173965454, 0.6612296104431152, 0.12842999398708344, 2.674046516418457, 0.4643377363681793], "max_p": 0.7015478014945984, "max_p_per_token": [0.9820936322212219, 0.4721456468105316, 0.9234086275100708, 0.2826605439186096, 0.6868688464164734, 0.6557666659355164, 0.998439610004425, 0.7306463122367859, 0.9998031258583069, 0.9458895325660706, 0.9771469235420227, 0.8006663918495178, 0.26261165738105774, 0.3802904486656189, 0.45757773518562317, 0.6147145628929138, 0.7909746170043945, 0.971969485282898, 0.20626601576805115, 0.8910157680511475], "n_positions_probed": 1, "per_restart_best": [10.064926147460938]}
+{"step": 107, "discrete_loss": 11.182768821716309, "best_sample_loss": 10.216118812561035, "soft_loss": 8.312865257263184, "best_discrete": 10.064926147460938, "best_soft": 8.312865257263184, "best_argmax": 10.99799633026123, "best_sampling": 10.064926147460938, "relax_gap": 0.2566362240163575, "n_match": 5, "g_first_norm": 159.45578002929688, "vocab_size": 50257, "entropy": 0.9127325415611267, "entropy_per_token": [0.14139819145202637, 1.3730705976486206, 0.2889013886451721, 2.039424419403076, 0.7814611196517944, 1.0693902969360352, 0.013542955741286278, 1.0131911039352417, 0.0020888415165245533, 0.2749529778957367, 0.14112664759159088, 0.8753082156181335, 2.1816744804382324, 1.3118157386779785, 1.6853362321853638, 1.0375231504440308, 0.6572455167770386, 0.1311328113079071, 2.7538866996765137, 0.482178270816803], "max_p": 0.6889858841896057, "max_p_per_token": [0.9780539274215698, 0.4791383743286133, 0.9259088039398193, 0.28655126690864563, 0.6792564392089844, 0.5946766138076782, 0.998406708240509, 0.5947486162185669, 0.9998158812522888, 0.9459275007247925, 0.9779734015464783, 0.7992755770683289, 0.3177538514137268, 0.3609805107116699, 0.4556043744087219, 0.5911353826522827, 0.7946085929870605, 0.9712079763412476, 0.14346382021903992, 0.8852302432060242], "n_positions_probed": 1, "per_restart_best": [10.064926147460938]}
+{"step": 108, "discrete_loss": 10.747529983520508, "best_sample_loss": 10.140727043151855, "soft_loss": 8.417596817016602, "best_discrete": 10.064926147460938, "best_soft": 8.312865257263184, "best_argmax": 10.747529983520508, "best_sampling": 10.064926147460938, "relax_gap": 0.21678778008309432, "n_match": 6, "g_first_norm": 349.421875, "vocab_size": 50257, "entropy": 0.8875482678413391, "entropy_per_token": [0.1440836787223816, 1.3851776123046875, 0.2770768702030182, 2.040135145187378, 0.8323081731796265, 0.8812953233718872, 0.013810764998197556, 0.986865758895874, 0.0027083922177553177, 0.27377867698669434, 0.13698315620422363, 0.917603075504303, 2.2901439666748047, 1.2197189331054688, 1.48019540309906, 1.026884913444519, 0.6371828317642212, 0.14248670637607574, 2.6714720726013184, 0.3910537362098694], "max_p": 0.7064283490180969, "max_p_per_token": [0.9776424169540405, 0.4765010178089142, 0.9300777912139893, 0.2784159779548645, 0.6386740207672119, 0.7063974142074585, 0.9983647465705872, 0.6212073564529419, 0.9997544884681702, 0.9462931752204895, 0.9788770079612732, 0.7885056734085083, 0.25331056118011475, 0.46441322565078735, 0.5745828747749329, 0.5868035554885864, 0.8036661148071289, 0.9679555892944336, 0.22359280288219452, 0.9135308861732483], "n_positions_probed": 1, "per_restart_best": [10.064926147460938]}
+{"step": 109, "discrete_loss": 10.704617500305176, "best_sample_loss": 10.093263626098633, "soft_loss": 8.532597541809082, "best_discrete": 10.064926147460938, "best_soft": 8.312865257263184, "best_argmax": 10.704617500305176, "best_sampling": 10.064926147460938, "relax_gap": 0.20290495745730028, "n_match": 6, "g_first_norm": 144.62744140625, "vocab_size": 50257, "entropy": 0.9194955825805664, "entropy_per_token": [0.16687697172164917, 1.3765441179275513, 0.2724839448928833, 2.029235363006592, 0.8501806259155273, 0.9143471717834473, 0.013699153438210487, 0.993079662322998, 0.0025868036318570375, 0.5847678184509277, 0.1357612907886505, 0.9200630187988281, 2.254028797149658, 1.2779359817504883, 1.5213074684143066, 1.049234390258789, 0.650658130645752, 0.14540743827819824, 2.807006597518921, 0.4247083067893982], "max_p": 0.6898674368858337, "max_p_per_token": [0.9733775854110718, 0.4768078923225403, 0.9316166639328003, 0.2876415550708771, 0.6198145151138306, 0.6870244741439819, 0.9983797073364258, 0.6185880303382874, 0.9997664093971252, 0.8441389203071594, 0.9791552424430847, 0.786243736743927, 0.25796836614608765, 0.40704065561294556, 0.5552048087120056, 0.5600606799125671, 0.7982401847839355, 0.9671016931533813, 0.14587071537971497, 0.9033060669898987], "n_positions_probed": 1, "per_restart_best": [10.064926147460938]}
+{"step": 110, "discrete_loss": 11.182768821716309, "best_sample_loss": 10.163442611694336, "soft_loss": 8.319419860839844, "best_discrete": 10.064926147460938, "best_soft": 8.312865257263184, "best_argmax": 10.704617500305176, "best_sampling": 10.064926147460938, "relax_gap": 0.25605008978778154, "n_match": 5, "g_first_norm": 204.2069091796875, "vocab_size": 50257, "entropy": 0.9371695518493652, "entropy_per_token": [0.2239707112312317, 1.3673425912857056, 0.2664426565170288, 2.0220935344696045, 0.8598588705062866, 1.0776443481445312, 0.013847490772604942, 0.9922441244125366, 0.0024515630211681128, 0.6069061160087585, 0.21068936586380005, 0.9146915078163147, 2.177870273590088, 1.3363970518112183, 1.5912922620773315, 1.051647663116455, 0.6522626280784607, 0.1490871012210846, 2.7871909141540527, 0.43945902585983276], "max_p": 0.6809487342834473, "max_p_per_token": [0.9621546864509583, 0.47711676359176636, 0.9336387515068054, 0.2934300899505615, 0.6110191941261292, 0.5805009603500366, 0.9983583092689514, 0.6213558912277222, 0.9997797608375549, 0.834257185459137, 0.9637990593910217, 0.7867266535758972, 0.3103039860725403, 0.33965644240379333, 0.5156335234642029, 0.5501378774642944, 0.7982450723648071, 0.966016948223114, 0.17809221148490906, 0.8987508416175842], "n_positions_probed": 1, "per_restart_best": [10.064926147460938]}
+{"step": 111, "discrete_loss": 11.234583854675293, "best_sample_loss": 10.123961448669434, "soft_loss": 8.222587585449219, "best_discrete": 10.064926147460938, "best_soft": 8.222587585449219, "best_argmax": 10.704617500305176, "best_sampling": 10.064926147460938, "relax_gap": 0.26810038611021864, "n_match": 5, "g_first_norm": 160.37460327148438, "vocab_size": 50257, "entropy": 0.9260196685791016, "entropy_per_token": [0.2396639883518219, 1.358467698097229, 0.26538515090942383, 1.981194019317627, 0.83345627784729, 0.9850918054580688, 0.013769099488854408, 0.9867610931396484, 0.0022872393019497395, 0.6320618391036987, 0.20556184649467468, 0.6555268168449402, 2.241515874862671, 1.3654043674468994, 1.6271438598632812, 1.0417815446853638, 0.6593042612075806, 0.15399545431137085, 2.8359436988830566, 0.4360767602920532], "max_p": 0.6818755865097046, "max_p_per_token": [0.9590347409248352, 0.4781917929649353, 0.9340274333953857, 0.31594109535217285, 0.6329766511917114, 0.6405499577522278, 0.9983660578727722, 0.6274012327194214, 0.9997958540916443, 0.8225069642066956, 0.9649268388748169, 0.7802942991256714, 0.27800092101097107, 0.34753406047821045, 0.49172574281692505, 0.5533266663551331, 0.7965322732925415, 0.96455317735672, 0.15201924741268158, 0.899806022644043], "n_positions_probed": 1, "per_restart_best": [10.064926147460938]}
+{"step": 112, "discrete_loss": 10.68055534362793, "best_sample_loss": 10.054859161376953, "soft_loss": 8.564443588256836, "best_discrete": 10.054859161376953, "best_soft": 8.222587585449219, "best_argmax": 10.68055534362793, "best_sampling": 10.054859161376953, "relax_gap": 0.19812750248362096, "n_match": 6, "g_first_norm": 683.7437133789062, "vocab_size": 50257, "entropy": 0.8701723217964172, "entropy_per_token": [0.1890883594751358, 1.3596863746643066, 0.26006343960762024, 1.970494031906128, 0.8161542415618896, 0.7431957721710205, 0.01617990806698799, 0.9011111855506897, 0.001996356062591076, 0.6483826041221619, 0.20998170971870422, 0.7388949990272522, 2.2537050247192383, 1.0558432340621948, 1.2764555215835571, 0.9835209846496582, 0.6340013742446899, 0.17729830741882324, 2.8226022720336914, 0.34478968381881714], "max_p": 0.7144767642021179, "max_p_per_token": [0.969258189201355, 0.4745391607284546, 0.9357504844665527, 0.310465931892395, 0.6361144781112671, 0.7717287540435791, 0.9980189800262451, 0.6814203262329102, 0.9998238682746887, 0.8170592188835144, 0.9635825753211975, 0.7081275582313538, 0.3283352255821228, 0.5784538984298706, 0.6513175964355469, 0.5935925841331482, 0.8036714196205139, 0.9573596119880676, 0.18460264801979065, 0.9263136982917786], "n_positions_probed": 1, "per_restart_best": [10.054859161376953]}
+{"step": 113, "discrete_loss": 10.513526916503906, "best_sample_loss": 10.136056900024414, "soft_loss": 8.396576881408691, "best_discrete": 10.054859161376953, "best_soft": 8.222587585449219, "best_argmax": 10.513526916503906, "best_sampling": 10.054859161376953, "relax_gap": 0.20135488803210963, "n_match": 6, "g_first_norm": 151.7500762939453, "vocab_size": 50257, "entropy": 0.8907537460327148, "entropy_per_token": [0.21380771696567535, 1.3583970069885254, 0.25342997908592224, 1.9865570068359375, 0.8414334058761597, 0.727811336517334, 0.016247566789388657, 0.8943607807159424, 0.0018629271071404219, 0.6523888111114502, 0.21791931986808777, 0.7408651113510132, 2.2619824409484863, 1.1986443996429443, 1.2894072532653809, 1.0055458545684814, 0.653762698173523, 0.1809186190366745, 2.92870831489563, 0.3910246789455414], "max_p": 0.7041105628013611, "max_p_per_token": [0.9643236398696899, 0.46972739696502686, 0.9379792213439941, 0.30301934480667114, 0.6052972674369812, 0.7784866094589233, 0.9980085492134094, 0.6857856512069702, 0.999836802482605, 0.8148992657661438, 0.961858332157135, 0.6993114948272705, 0.30190742015838623, 0.5475190281867981, 0.6481771469116211, 0.5691539645195007, 0.7947337031364441, 0.9562017917633057, 0.13304968178272247, 0.9129353761672974], "n_positions_probed": 1, "per_restart_best": [10.054859161376953]}
+{"step": 114, "discrete_loss": 10.68055534362793, "best_sample_loss": 10.13383674621582, "soft_loss": 8.270600318908691, "best_discrete": 10.054859161376953, "best_soft": 8.222587585449219, "best_argmax": 10.513526916503906, "best_sampling": 10.054859161376953, "relax_gap": 0.2256394866355923, "n_match": 6, "g_first_norm": 157.6834716796875, "vocab_size": 50257, "entropy": 0.8979887366294861, "entropy_per_token": [0.2356891632080078, 1.3576807975769043, 0.25085699558258057, 1.9757270812988281, 0.8408849835395813, 0.7040102481842041, 0.016484742984175682, 0.8893842697143555, 0.0017292806878685951, 0.6579250693321228, 0.22896376252174377, 0.7350075840950012, 2.2517037391662598, 1.2448558807373047, 1.3554224967956543, 1.0116496086120605, 0.6662676334381104, 0.18502689898014069, 2.9102113246917725, 0.44029247760772705], "max_p": 0.7002987265586853, "max_p_per_token": [0.9598190188407898, 0.4635028839111328, 0.9388923645019531, 0.3081466257572174, 0.5979454517364502, 0.7890400886535645, 0.9979737401008606, 0.6890219449996948, 0.9998495578765869, 0.8121623992919922, 0.9594337940216064, 0.6955997943878174, 0.2825649082660675, 0.5121825337409973, 0.6364095211029053, 0.5604645013809204, 0.788922905921936, 0.9548774361610413, 0.16128231585025787, 0.897883415222168], "n_positions_probed": 1, "per_restart_best": [10.054859161376953]}
+{"step": 115, "discrete_loss": 10.68055534362793, "best_sample_loss": 10.054859161376953, "soft_loss": 8.188240051269531, "best_discrete": 10.054859161376953, "best_soft": 8.188240051269531, "best_argmax": 10.513526916503906, "best_sampling": 10.054859161376953, "relax_gap": 0.23335072120995334, "n_match": 6, "g_first_norm": 156.71835327148438, "vocab_size": 50257, "entropy": 0.8560888171195984, "entropy_per_token": [0.2647436857223511, 1.35880446434021, 0.24886086583137512, 1.9688959121704102, 0.8373278379440308, 0.6827813982963562, 0.01667032018303871, 0.8849211931228638, 0.0016007761005312204, 0.6665444374084473, 0.2403249740600586, 0.7335027456283569, 2.2503204345703125, 1.307603359222412, 1.3208038806915283, 0.015550926327705383, 0.6814512014389038, 0.18995881080627441, 2.9489293098449707, 0.5021794438362122], "max_p": 0.7155241966247559, "max_p_per_token": [0.9536536335945129, 0.45520174503326416, 0.9396035075187683, 0.31233108043670654, 0.5914366841316223, 0.7982220649719238, 0.9979464411735535, 0.6918560266494751, 0.9998618364334106, 0.8079879879951477, 0.9568731188774109, 0.6875004768371582, 0.26449936628341675, 0.4535655081272125, 0.6507917642593384, 0.9980870485305786, 0.7820586562156677, 0.9532710313796997, 0.13784684240818024, 0.877888560295105], "n_positions_probed": 1, "per_restart_best": [10.054859161376953]}
+{"step": 116, "discrete_loss": 10.68055534362793, "best_sample_loss": 10.188763618469238, "soft_loss": 8.160421371459961, "best_discrete": 10.054859161376953, "best_soft": 8.160421371459961, "best_argmax": 10.513526916503906, "best_sampling": 10.054859161376953, "relax_gap": 0.23595533107475472, "n_match": 6, "g_first_norm": 166.9472198486328, "vocab_size": 50257, "entropy": 0.8655157089233398, "entropy_per_token": [0.30955642461776733, 1.3606493473052979, 0.24597758054733276, 1.968517780303955, 0.8376943469047546, 0.6619563102722168, 0.016874581575393677, 0.8772112727165222, 0.0014637617859989405, 0.675415575504303, 0.2518741488456726, 0.7338452339172363, 2.264542818069458, 1.3702735900878906, 1.2913132905960083, 0.01730627566576004, 0.6925718784332275, 0.19320154190063477, 2.9623141288757324, 0.5777541399002075], "max_p": 0.7095746397972107, "max_p_per_token": [0.9437701106071472, 0.44711244106292725, 0.9406476020812988, 0.3143002986907959, 0.5778804421424866, 0.8070447444915771, 0.9979164004325867, 0.6965855956077576, 0.9998748302459717, 0.8034120798110962, 0.9542211890220642, 0.6750415563583374, 0.25038960576057434, 0.3949874937534332, 0.6623943448066711, 0.997835099697113, 0.7801234126091003, 0.9522086977958679, 0.14392879605293274, 0.8518190979957581], "n_positions_probed": 1, "per_restart_best": [10.054859161376953]}
+{"step": 117, "discrete_loss": 10.68055534362793, "best_sample_loss": 10.137833595275879, "soft_loss": 8.082110404968262, "best_discrete": 10.054859161376953, "best_soft": 8.082110404968262, "best_argmax": 10.513526916503906, "best_sampling": 10.054859161376953, "relax_gap": 0.24328743731569283, "n_match": 6, "g_first_norm": 168.55821228027344, "vocab_size": 50257, "entropy": 0.8665637969970703, "entropy_per_token": [0.3526363670825958, 1.3627879619598389, 0.24314728379249573, 1.9576188325881958, 0.8337914347648621, 0.6386181712150574, 0.0171048641204834, 0.8698434233665466, 0.0013214604696258903, 0.6825754046440125, 0.26358258724212646, 0.73396235704422, 2.2677152156829834, 1.4219790697097778, 1.2557573318481445, 0.019188418984413147, 0.7001206874847412, 0.007830959744751453, 2.9860126972198486, 0.7156811952590942], "max_p": 0.7065459489822388, "max_p_per_token": [0.9338370561599731, 0.4385051131248474, 0.9416738748550415, 0.3207620084285736, 0.567965030670166, 0.816740870475769, 0.9978830218315125, 0.7009872198104858, 0.9998881816864014, 0.7994966506958008, 0.9514433741569519, 0.661282479763031, 0.2453496903181076, 0.3684464693069458, 0.6755917072296143, 0.9975590705871582, 0.7780512571334839, 0.9990147352218628, 0.13683682680130005, 0.7996035218238831], "n_positions_probed": 1, "per_restart_best": [10.054859161376953]}
+{"step": 118, "discrete_loss": 11.143234252929688, "best_sample_loss": 10.054859161376953, "soft_loss": 7.975900173187256, "best_discrete": 10.054859161376953, "best_soft": 7.975900173187256, "best_argmax": 10.513526916503906, "best_sampling": 10.054859161376953, "relax_gap": 0.2842383106959905, "n_match": 7, "g_first_norm": 167.31613159179688, "vocab_size": 50257, "entropy": 0.7875178456306458, "entropy_per_token": [0.3873637020587921, 1.361743688583374, 0.2402595430612564, 1.9584903717041016, 0.8282501697540283, 0.6180367469787598, 0.017230158671736717, 0.8654569387435913, 0.0011877636425197124, 0.6911950707435608, 0.27468523383140564, 0.720168948173523, 2.267821788787842, 1.4639817476272583, 1.2155486345291138, 0.021057505160570145, 0.7100537419319153, 0.007984388619661331, 1.1705609560012817, 0.9292796850204468], "max_p": 0.7326371073722839, "max_p_per_token": [0.9253516793251038, 0.43653932213783264, 0.9427313208580017, 0.31939151883125305, 0.5604830384254456, 0.8249958157539368, 0.9978659749031067, 0.7035213112831116, 0.9999003410339355, 0.7947189211845398, 0.9487360119819641, 0.664342999458313, 0.2458174228668213, 0.34156227111816406, 0.6898865699768066, 0.9972772002220154, 0.7747044563293457, 0.9989927411079407, 0.7816792726516724, 0.7042444944381714], "n_positions_probed": 1, "per_restart_best": [10.054859161376953]}
+{"step": 119, "discrete_loss": 11.213569641113281, "best_sample_loss": 10.092350959777832, "soft_loss": 9.405500411987305, "best_discrete": 10.054859161376953, "best_soft": 7.975900173187256, "best_argmax": 10.513526916503906, "best_sampling": 10.054859161376953, "relax_gap": 0.1612393989597118, "n_match": 7, "g_first_norm": 263.2294921875, "vocab_size": 50257, "entropy": 0.8595771789550781, "entropy_per_token": [0.503480076789856, 1.3674697875976562, 0.24185895919799805, 2.01609468460083, 0.8409489989280701, 0.622826874256134, 0.01780136302113533, 0.8556406497955322, 0.001126896939240396, 0.6950839757919312, 0.2825598418712616, 0.7344585061073303, 2.228536605834961, 1.4807454347610474, 1.2996139526367188, 0.02176949381828308, 0.7076760530471802, 0.007938019931316376, 2.3778862953186035, 0.8880270719528198], "max_p": 0.7062050104141235, "max_p_per_token": [0.8946887254714966, 0.4282241463661194, 0.9422286748886108, 0.27944859862327576, 0.5179376602172852, 0.8224358558654785, 0.997779905796051, 0.7086697816848755, 0.9999059438705444, 0.791283905506134, 0.9467845559120178, 0.6363011002540588, 0.25135940313339233, 0.3710706830024719, 0.655125081539154, 0.9971613883972168, 0.7798987627029419, 0.9989995360374451, 0.3989056646823883, 0.7058905363082886], "n_positions_probed": 1, "per_restart_best": [10.054859161376953]}
+{"step": 120, "discrete_loss": 11.028862953186035, "best_sample_loss": 10.119741439819336, "soft_loss": 8.958372116088867, "best_discrete": 10.054859161376953, "best_soft": 7.975900173187256, "best_argmax": 10.513526916503906, "best_sampling": 10.054859161376953, "relax_gap": 0.18773384399513654, "n_match": 6, "g_first_norm": 368.1870422363281, "vocab_size": 50257, "entropy": 0.8841323852539062, "entropy_per_token": [0.663153886795044, 1.3906593322753906, 0.2573835253715515, 1.8835325241088867, 0.8280538320541382, 0.616861879825592, 0.01944540999829769, 0.8528755903244019, 0.0010907381074503064, 0.7107642292976379, 0.3128969669342041, 0.740433931350708, 2.3733930587768555, 1.3754183053970337, 1.3446024656295776, 0.022427616640925407, 0.688512921333313, 0.008462022058665752, 2.8771259784698486, 0.7155516147613525], "max_p": 0.6983057260513306, "max_p_per_token": [0.8489797115325928, 0.4168652296066284, 0.9372620582580566, 0.34041866660118103, 0.4939480423927307, 0.8241497278213501, 0.9975336790084839, 0.7088546752929688, 0.9999090433120728, 0.7807292342185974, 0.9385107159614563, 0.612360954284668, 0.2401275485754013, 0.48520615696907043, 0.6304462552070618, 0.9970404505729675, 0.7906174659729004, 0.9989239573478699, 0.14906641840934753, 0.7751637101173401], "n_positions_probed": 1, "per_restart_best": [10.054859161376953]}
+{"step": 121, "discrete_loss": 10.075034141540527, "best_sample_loss": 10.054859161376953, "soft_loss": 7.9596123695373535, "best_discrete": 10.054859161376953, "best_soft": 7.9596123695373535, "best_argmax": 10.075034141540527, "best_sampling": 10.054859161376953, "relax_gap": 0.2099667100164997, "n_match": 6, "g_first_norm": 483.5604553222656, "vocab_size": 50257, "entropy": 0.872891902923584, "entropy_per_token": [0.6898394823074341, 0.871034562587738, 0.2568167448043823, 1.9527314901351929, 0.8096553087234497, 0.5782981514930725, 0.020041514188051224, 0.8600724935531616, 0.0010524861281737685, 0.7369893789291382, 0.3110509216785431, 0.7312678098678589, 2.4044365882873535, 1.4649839401245117, 1.2539559602737427, 0.024476980790495872, 0.6686298847198486, 0.009036676958203316, 2.8878438472747803, 0.9256243705749512], "max_p": 0.7119756937026978, "max_p_per_token": [0.8416133522987366, 0.7644397616386414, 0.9374666213989258, 0.3251018226146698, 0.513045608997345, 0.8399147391319275, 0.9974489808082581, 0.7035496234893799, 0.9999125003814697, 0.7658935189247131, 0.9390120506286621, 0.6090927720069885, 0.22488582134246826, 0.41933202743530273, 0.6618248820304871, 0.9967126846313477, 0.8004509806632996, 0.9988400340080261, 0.1748526692390442, 0.7261232137680054], "n_positions_probed": 1, "per_restart_best": [10.054859161376953]}
+{"step": 122, "discrete_loss": 10.075034141540527, "best_sample_loss": 10.053330421447754, "soft_loss": 7.831681728363037, "best_discrete": 10.053330421447754, "best_soft": 7.831681728363037, "best_argmax": 10.075034141540527, "best_sampling": 10.053330421447754, "relax_gap": 0.22266449737652894, "n_match": 5, "g_first_norm": 161.75682067871094, "vocab_size": 50257, "entropy": 0.8810624480247498, "entropy_per_token": [0.6218337416648865, 0.9225172996520996, 0.25937578082084656, 1.9045886993408203, 0.7995734214782715, 0.5796436071395874, 0.0196915902197361, 0.8689264059066772, 0.0010120976949110627, 0.7610187530517578, 0.3133182227611542, 0.7256671190261841, 2.3516268730163574, 1.5708427429199219, 1.2362831830978394, 0.02521687000989914, 0.6915103197097778, 0.008751587942242622, 2.979231119155884, 0.9806185960769653], "max_p": 0.7074589729309082, "max_p_per_token": [0.8617743253707886, 0.7446871995925903, 0.9369269609451294, 0.34803083539009094, 0.5435909032821655, 0.8388316631317139, 0.9975019097328186, 0.6976111531257629, 0.9999161958694458, 0.7508484721183777, 0.9383381605148315, 0.608907163143158, 0.23791588842868805, 0.36724233627319336, 0.6693428754806519, 0.9965887069702148, 0.793263852596283, 0.998881995677948, 0.12018778175115585, 0.6987910866737366], "n_positions_probed": 1, "per_restart_best": [10.053330421447754]}
+{"step": 123, "discrete_loss": 10.075034141540527, "best_sample_loss": 10.094446182250977, "soft_loss": 7.576960563659668, "best_discrete": 10.053330421447754, "best_soft": 7.576960563659668, "best_argmax": 10.075034141540527, "best_sampling": 10.053330421447754, "relax_gap": 0.24794690943835257, "n_match": 5, "g_first_norm": 157.34542846679688, "vocab_size": 50257, "entropy": 0.8861163258552551, "entropy_per_token": [0.5714513063430786, 0.9592675566673279, 0.25832805037498474, 1.9034329652786255, 0.7868346571922302, 0.5816994905471802, 0.019564950838685036, 0.8746454119682312, 0.0009731148602440953, 0.7826800346374512, 0.3187485337257385, 0.7200089693069458, 2.320477247238159, 1.5950744152069092, 1.229050636291504, 0.026316527277231216, 0.7135014533996582, 0.008460751734673977, 3.0012972354888916, 1.0505131483078003], "max_p": 0.7031236886978149, "max_p_per_token": [0.8760817646980286, 0.730373740196228, 0.9373273253440857, 0.35579580068588257, 0.5677372813224792, 0.8372815251350403, 0.9975215792655945, 0.6936628818511963, 0.9999196529388428, 0.7368036508560181, 0.9368910789489746, 0.6078538298606873, 0.24655799567699432, 0.298789381980896, 0.6726161241531372, 0.996407687664032, 0.7857248187065125, 0.9989246726036072, 0.12661652266979218, 0.6595854163169861], "n_positions_probed": 1, "per_restart_best": [10.053330421447754]}
+{"step": 124, "discrete_loss": 9.729219436645508, "best_sample_loss": 9.938675880432129, "soft_loss": 7.381900787353516, "best_discrete": 9.729219436645508, "best_soft": 7.381900787353516, "best_argmax": 9.729219436645508, "best_sampling": 9.938675880432129, "relax_gap": 0.24126484807719717, "n_match": 20, "g_first_norm": 153.0539093017578, "vocab_size": 50257, "entropy": 0.9062259793281555, "entropy_per_token": [0.5456128120422363, 0.9777730107307434, 0.2549148499965668, 1.8946177959442139, 1.1403799057006836, 0.5824773907661438, 0.01980840228497982, 0.8786355257034302, 0.0009289926965720952, 0.803941547870636, 0.32627782225608826, 0.7188282012939453, 2.305633544921875, 1.5421029329299927, 1.231758952140808, 0.02770066075026989, 0.7384947538375854, 0.008166169747710228, 3.024156332015991, 1.1023099422454834], "max_p": 0.6947150230407715, "max_p_per_token": [0.883163571357727, 0.7234956622123718, 0.9385181069374084, 0.35580962896347046, 0.38531428575515747, 0.8360536098480225, 0.99748694896698, 0.6910203099250793, 0.9999237060546875, 0.7227057814598083, 0.9348668456077576, 0.6000921130180359, 0.246064692735672, 0.39500120282173157, 0.6720399856567383, 0.9961787462234497, 0.7762361168861389, 0.9989676475524902, 0.11766213178634644, 0.6236985921859741], "n_positions_probed": 1, "per_restart_best": [9.729219436645508]}
+{"step": 125, "discrete_loss": 9.98533821105957, "best_sample_loss": 9.454680442810059, "soft_loss": 7.350588798522949, "best_discrete": 9.454680442810059, "best_soft": 7.350588798522949, "best_argmax": 9.729219436645508, "best_sampling": 9.454680442810059, "relax_gap": 0.26386180987023783, "n_match": 17, "g_first_norm": 155.4501495361328, "vocab_size": 50257, "entropy": 0.9080042839050293, "entropy_per_token": [0.5406767129898071, 0.97611403465271, 0.2503717243671417, 1.8828105926513672, 1.1435798406600952, 0.5864448547363281, 0.020225631073117256, 0.8791855573654175, 0.000881597981788218, 0.8296719193458557, 0.3311958312988281, 0.7152260541915894, 2.2898685932159424, 1.5198137760162354, 1.2268915176391602, 0.0290830135345459, 0.762316107749939, 0.0078545231372118, 3.0347933769226074, 1.1330807209014893], "max_p": 0.6937068104743958, "max_p_per_token": [0.8844655156135559, 0.7251072525978088, 0.9400400519371033, 0.3552473783493042, 0.3724726140499115, 0.8357502818107605, 0.997425377368927, 0.6908132433891296, 0.9999279975891113, 0.7058391571044922, 0.9335052371025085, 0.5975101590156555, 0.2463776171207428, 0.4341503977775574, 0.6742720007896423, 0.9959477782249451, 0.7675644755363464, 0.9990127086639404, 0.12205631285905838, 0.5966509580612183], "n_positions_probed": 1, "per_restart_best": [9.454680442810059]}
+{"step": 126, "discrete_loss": 9.98533821105957, "best_sample_loss": 9.456230163574219, "soft_loss": 7.289572238922119, "best_discrete": 9.454680442810059, "best_soft": 7.289572238922119, "best_argmax": 9.729219436645508, "best_sampling": 9.454680442810059, "relax_gap": 0.2699724250853789, "n_match": 17, "g_first_norm": 151.70166015625, "vocab_size": 50257, "entropy": 0.9106548428535461, "entropy_per_token": [0.5409128665924072, 0.9716241955757141, 0.2453463226556778, 1.8731951713562012, 1.143432855606079, 0.5812801718711853, 0.01719048246741295, 0.8796420693397522, 0.0008355987374670804, 0.85563063621521, 0.33557674288749695, 0.7150874733924866, 2.2912850379943848, 1.4984405040740967, 1.223233938217163, 0.030697930604219437, 0.7893620133399963, 0.0075484588742256165, 3.049217700958252, 1.163557767868042], "max_p": 0.6924823522567749, "max_p_per_token": [0.8843342661857605, 0.7278860211372375, 0.94169682264328, 0.3538687229156494, 0.3826623260974884, 0.8365679383277893, 0.9977624416351318, 0.690751850605011, 0.999932050704956, 0.6882269382476807, 0.9322422742843628, 0.5889301300048828, 0.25077229738235474, 0.46334308385849, 0.6758560538291931, 0.9956744313240051, 0.7571461796760559, 0.9990567564964294, 0.11700130254030228, 0.5659347772598267], "n_positions_probed": 1, "per_restart_best": [9.454680442810059]}
+{"step": 127, "discrete_loss": 9.98533821105957, "best_sample_loss": 9.41080379486084, "soft_loss": 7.235768795013428, "best_discrete": 9.41080379486084, "best_soft": 7.235768795013428, "best_argmax": 9.729219436645508, "best_sampling": 9.41080379486084, "relax_gap": 0.2753606696066411, "n_match": 16, "g_first_norm": 151.01246643066406, "vocab_size": 50257, "entropy": 0.91355961561203, "entropy_per_token": [0.5451037883758545, 0.9657776355743408, 0.23999394476413727, 1.8638560771942139, 1.140592098236084, 0.575562059879303, 0.01759442873299122, 0.8807089328765869, 0.0007935730391182005, 0.8819707632064819, 0.3391820192337036, 0.715375542640686, 2.2934207916259766, 1.476081371307373, 1.2205092906951904, 0.032450947910547256, 0.8182307481765747, 0.007247475441545248, 3.0655031204223633, 1.1912381649017334], "max_p": 0.6909433603286743, "max_p_per_token": [0.883095383644104, 0.7312650680541992, 0.943439781665802, 0.352417916059494, 0.3932039737701416, 0.837543249130249, 0.9977015852928162, 0.6906616687774658, 0.9999356269836426, 0.6697185039520264, 0.9311714768409729, 0.5795674920082092, 0.25400838255882263, 0.48802363872528076, 0.6769000291824341, 0.9953728318214417, 0.7455049157142639, 0.999099850654602, 0.1173478364944458, 0.53288733959198], "n_positions_probed": 1, "per_restart_best": [9.41080379486084]}
+{"step": 128, "discrete_loss": 9.98533821105957, "best_sample_loss": 9.253216743469238, "soft_loss": 7.183878421783447, "best_discrete": 9.253216743469238, "best_soft": 7.183878421783447, "best_argmax": 9.729219436645508, "best_sampling": 9.253216743469238, "relax_gap": 0.280557326157794, "n_match": 15, "g_first_norm": 150.1456298828125, "vocab_size": 50257, "entropy": 0.9166662096977234, "entropy_per_token": [0.5509946346282959, 0.9596537947654724, 0.2343377321958542, 1.8549696207046509, 1.135184407234192, 0.5692857503890991, 0.017992865294218063, 0.8813750743865967, 0.0008066343143582344, 0.9081535339355469, 0.3420165181159973, 0.7165207862854004, 2.2980129718780518, 1.4541194438934326, 1.2178418636322021, 0.03439799323678017, 0.8507587909698486, 0.006954210810363293, 3.0895142555236816, 1.2104326486587524], "max_p": 0.6888694763183594, "max_p_per_token": [0.8813645839691162, 0.7347368597984314, 0.9452565312385559, 0.3505396842956543, 0.4045797288417816, 0.8386947512626648, 0.9976415634155273, 0.6905609965324402, 0.9999350309371948, 0.6504706144332886, 0.9302951097488403, 0.5679991245269775, 0.2560942769050598, 0.5089131593704224, 0.6777956485748291, 0.995032787322998, 0.7314968109130859, 0.9991414546966553, 0.1134144738316536, 0.5034264922142029], "n_positions_probed": 1, "per_restart_best": [9.253216743469238]}
+{"step": 129, "discrete_loss": 9.98533821105957, "best_sample_loss": 9.242486953735352, "soft_loss": 7.1343183517456055, "best_discrete": 9.242486953735352, "best_soft": 7.1343183517456055, "best_argmax": 9.729219436645508, "best_sampling": 9.242486953735352, "relax_gap": 0.28552061022392106, "n_match": 14, "g_first_norm": 150.9859161376953, "vocab_size": 50257, "entropy": 0.9197036027908325, "entropy_per_token": [0.5584930777549744, 0.954814076423645, 0.22853422164916992, 1.8466168642044067, 1.1275650262832642, 0.5626987218856812, 0.018365882337093353, 0.8820849061012268, 0.0007693162187933922, 0.9338045120239258, 0.3439904451370239, 0.7178733944892883, 2.301908493041992, 1.43402099609375, 1.2154414653778076, 0.0365249402821064, 0.8858246803283691, 0.006663801148533821, 3.11860728263855, 1.2194702625274658], "max_p": 0.686890721321106, "max_p_per_token": [0.8791582584381104, 0.7376789450645447, 0.9470981955528259, 0.3480176031589508, 0.41649842262268066, 0.8399226069450378, 0.997585654258728, 0.6904679536819458, 0.9999383687973022, 0.631085991859436, 0.929658830165863, 0.5549312829971313, 0.25821781158447266, 0.5262768268585205, 0.6784765720367432, 0.9946557283401489, 0.7151427268981934, 0.9991825222969055, 0.1114983856678009, 0.4823213517665863], "n_positions_probed": 1, "per_restart_best": [9.242486953735352]}
+{"step": 130, "discrete_loss": 9.98533821105957, "best_sample_loss": 9.365426063537598, "soft_loss": 7.086443901062012, "best_discrete": 9.242486953735352, "best_soft": 7.086443901062012, "best_argmax": 9.729219436645508, "best_sampling": 9.242486953735352, "relax_gap": 0.2903150848497849, "n_match": 14, "g_first_norm": 152.84088134765625, "vocab_size": 50257, "entropy": 0.922928512096405, "entropy_per_token": [0.566328763961792, 0.9519519805908203, 0.222662091255188, 1.8388912677764893, 1.1178462505340576, 0.5559670925140381, 0.018717020750045776, 0.8824424743652344, 0.0007346441270783544, 0.9567412734031677, 0.34573525190353394, 0.7196893095970154, 2.307495594024658, 1.4161429405212402, 1.2131344079971313, 0.03886573761701584, 0.9231786727905273, 0.0063747623935341835, 3.154684066772461, 1.22098708152771], "max_p": 0.684787929058075, "max_p_per_token": [0.87681645154953, 0.7397850155830383, 0.9489395022392273, 0.34455516934394836, 0.4295564293861389, 0.8411629796028137, 0.997532844543457, 0.690631091594696, 0.999941349029541, 0.6122137308120728, 0.9292084574699402, 0.5376550555229187, 0.26041528582572937, 0.5408329367637634, 0.6790558695793152, 0.994234025478363, 0.6959406733512878, 0.999222993850708, 0.10738610476255417, 0.47067245841026306], "n_positions_probed": 1, "per_restart_best": [9.242486953735352]}
+{"step": 131, "discrete_loss": 9.98533821105957, "best_sample_loss": 9.253519058227539, "soft_loss": 7.037164211273193, "best_discrete": 9.242486953735352, "best_soft": 7.037164211273193, "best_argmax": 9.729219436645508, "best_sampling": 9.242486953735352, "relax_gap": 0.295250289721888, "n_match": 14, "g_first_norm": 157.14837646484375, "vocab_size": 50257, "entropy": 0.9265617728233337, "entropy_per_token": [0.5741963386535645, 0.9515276551246643, 0.21677260100841522, 1.831451416015625, 1.1061017513275146, 0.5491361021995544, 0.01902548223733902, 0.8817926645278931, 0.0007021059864200652, 0.9771096706390381, 0.3457549214363098, 0.7324730157852173, 2.311880588531494, 1.3994946479797363, 1.2107598781585693, 0.04140207916498184, 0.9615757465362549, 0.0060804751701653, 3.1953868865966797, 1.2186102867126465], "max_p": 0.682525098323822, "max_p_per_token": [0.8744045495986938, 0.7408757209777832, 0.9507655501365662, 0.3402586579322815, 0.4439118802547455, 0.8423890471458435, 0.9974865913391113, 0.6914595365524292, 0.9999442100524902, 0.5940775871276855, 0.9291394352912903, 0.5115082263946533, 0.26457294821739197, 0.5537589192390442, 0.679620087146759, 0.9937688708305359, 0.6736789345741272, 0.9992638230323792, 0.1038655936717987, 0.46575111150741577], "n_positions_probed": 1, "per_restart_best": [9.242486953735352]}
+{"step": 132, "discrete_loss": 9.936659812927246, "best_sample_loss": 9.551196098327637, "soft_loss": 6.980935573577881, "best_discrete": 9.242486953735352, "best_soft": 6.980935573577881, "best_argmax": 9.729219436645508, "best_sampling": 9.242486953735352, "relax_gap": 0.29745651909146287, "n_match": 13, "g_first_norm": 164.45411682128906, "vocab_size": 50257, "entropy": 0.9331268668174744, "entropy_per_token": [0.5814234614372253, 0.9533063173294067, 0.21086540818214417, 1.8242508172988892, 1.0920641422271729, 0.5420411825180054, 0.01926039718091488, 0.8792027831077576, 0.0006712365429848433, 0.994377613067627, 0.344055712223053, 0.7310003042221069, 2.395218849182129, 1.3829206228256226, 1.207411766052246, 0.04415880888700485, 1.0008057355880737, 0.0057755098678171635, 3.2388978004455566, 1.2148276567459106], "max_p": 0.6829881072044373, "max_p_per_token": [0.8720769882202148, 0.740998387336731, 0.9525750279426575, 0.33501023054122925, 0.4602351188659668, 0.84366774559021, 0.9974516034126282, 0.69350665807724, 0.9999469518661499, 0.5766993165016174, 0.9295679330825806, 0.5230587124824524, 0.2848576307296753, 0.5660521984100342, 0.6805793046951294, 0.9932532906532288, 0.647443413734436, 0.9993059635162354, 0.09956899285316467, 0.46390655636787415], "n_positions_probed": 1, "per_restart_best": [9.242486953735352]}
+{"step": 133, "discrete_loss": 10.060467720031738, "best_sample_loss": 9.404273986816406, "soft_loss": 6.930616855621338, "best_discrete": 9.242486953735352, "best_soft": 6.930616855621338, "best_argmax": 9.729219436645508, "best_sampling": 9.242486953735352, "relax_gap": 0.31110391201578513, "n_match": 12, "g_first_norm": 173.09051513671875, "vocab_size": 50257, "entropy": 0.9460671544075012, "entropy_per_token": [0.5880215764045715, 0.9605326056480408, 0.20331013202667236, 1.8201301097869873, 1.0767403841018677, 0.5346511006355286, 0.019487623125314713, 0.8761395215988159, 0.0006416764808818698, 1.0159939527511597, 0.3400956094264984, 0.731791615486145, 2.3553950786590576, 1.6072478294372559, 1.2081704139709473, 0.04679742082953453, 1.0435470342636108, 0.005432716105133295, 3.2808361053466797, 1.206380844116211], "max_p": 0.676427960395813, "max_p_per_token": [0.869830310344696, 0.7387000918388367, 0.9548504948616028, 0.32855281233787537, 0.4768858551979065, 0.8449925780296326, 0.9974184036254883, 0.6959614157676697, 0.9999494552612305, 0.5551924109458923, 0.9305875301361084, 0.5167545676231384, 0.3110140562057495, 0.4501950740814209, 0.6799832582473755, 0.9927482008934021, 0.6142688393592834, 0.999352753162384, 0.09831786900758743, 0.4730025827884674], "n_positions_probed": 1, "per_restart_best": [9.242486953735352]}
+{"step": 134, "discrete_loss": 10.474390029907227, "best_sample_loss": 9.866311073303223, "soft_loss": 6.887118339538574, "best_discrete": 9.242486953735352, "best_soft": 6.887118339538574, "best_argmax": 9.729219436645508, "best_sampling": 9.242486953735352, "relax_gap": 0.34248024754911915, "n_match": 12, "g_first_norm": 181.73025512695312, "vocab_size": 50257, "entropy": 0.9001731872558594, "entropy_per_token": [0.5896446108818054, 0.9626075029373169, 0.19576233625411987, 1.8171124458312988, 1.0575193166732788, 0.5249258279800415, 0.019803928211331367, 0.8650449514389038, 0.0006076739518903196, 1.0301746129989624, 0.33624520897865295, 0.728299617767334, 2.4222280979156494, 1.578493356704712, 0.2527455687522888, 0.0505567267537117, 1.056609034538269, 0.00509251281619072, 3.3025853633880615, 1.2074049711227417], "max_p": 0.6914434432983398, "max_p_per_token": [0.868829607963562, 0.7383583188056946, 0.9570959210395813, 0.32313576340675354, 0.49841490387916565, 0.8473060131072998, 0.9973704814910889, 0.7034185528755188, 0.9999524354934692, 0.5395960807800293, 0.9315800070762634, 0.5434802174568176, 0.29372021555900574, 0.4825732111930847, 0.9499659538269043, 0.9920185208320618, 0.595700204372406, 0.9993988275527954, 0.0995272621512413, 0.4674256145954132], "n_positions_probed": 1, "per_restart_best": [9.242486953735352]}
+{"step": 135, "discrete_loss": 10.684846878051758, "best_sample_loss": 9.379661560058594, "soft_loss": 7.707217216491699, "best_discrete": 9.242486953735352, "best_soft": 6.887118339538574, "best_argmax": 9.729219436645508, "best_sampling": 9.242486953735352, "relax_gap": 0.27867780376680423, "n_match": 11, "g_first_norm": 237.82276916503906, "vocab_size": 50257, "entropy": 0.8987706303596497, "entropy_per_token": [0.6247663497924805, 0.9497195482254028, 0.1826915591955185, 1.8335158824920654, 1.0500513315200806, 0.5172995924949646, 0.019880568608641624, 0.8648340702056885, 0.0005920998519286513, 1.1007959842681885, 0.3262934684753418, 0.7062038779258728, 2.330251693725586, 1.5677800178527832, 0.31703492999076843, 0.061597902327775955, 0.9999740719795227, 0.0048551964573562145, 3.3236746788024902, 1.1935993432998657], "max_p": 0.6965921521186829, "max_p_per_token": [0.8581942915916443, 0.7427883744239807, 0.9608566164970398, 0.3097408413887024, 0.5023636221885681, 0.8490908145904541, 0.9973594546318054, 0.7040745615959167, 0.9999538660049438, 0.4928162097930908, 0.9343369603157043, 0.6087368130683899, 0.32886457443237305, 0.4945548176765442, 0.9302449226379395, 0.9900996088981628, 0.6336475014686584, 0.9994305968284607, 0.10760333389043808, 0.48708420991897583], "n_positions_probed": 1, "per_restart_best": [9.242486953735352]}
+{"step": 136, "discrete_loss": 10.474390029907227, "best_sample_loss": 9.220468521118164, "soft_loss": 7.5966596603393555, "best_discrete": 9.220468521118164, "best_soft": 6.887118339538574, "best_argmax": 9.729219436645508, "best_sampling": 9.220468521118164, "relax_gap": 0.2747396613407721, "n_match": 11, "g_first_norm": 224.95358276367188, "vocab_size": 50257, "entropy": 0.9204420447349548, "entropy_per_token": [0.6770166158676147, 0.937602698802948, 0.17022883892059326, 1.8573354482650757, 1.0312620401382446, 0.5005556344985962, 0.020326200872659683, 0.8507524132728577, 0.0005855134222656488, 1.250597357749939, 0.3219400644302368, 0.7300204634666443, 2.487381935119629, 1.5448241233825684, 0.3802033066749573, 0.07329348474740982, 1.0501066446304321, 0.004676020238548517, 3.3307459354400635, 1.1893866062164307], "max_p": 0.6862528920173645, "max_p_per_token": [0.8411468863487244, 0.7470798492431641, 0.9643160104751587, 0.29277509450912476, 0.5216021537780762, 0.8550507426261902, 0.9972928166389465, 0.7134501934051514, 0.999954342842102, 0.43517062067985535, 0.9354676604270935, 0.5430801510810852, 0.25800779461860657, 0.5149595737457275, 0.9078481793403625, 0.9877592921257019, 0.6253032088279724, 0.9994544386863708, 0.0988200232386589, 0.48651769757270813], "n_positions_probed": 1, "per_restart_best": [9.220468521118164]}
+{"step": 137, "discrete_loss": 10.684846878051758, "best_sample_loss": 9.124794006347656, "soft_loss": 7.511502265930176, "best_discrete": 9.124794006347656, "best_soft": 6.887118339538574, "best_argmax": 9.729219436645508, "best_sampling": 9.124794006347656, "relax_gap": 0.2969948608847261, "n_match": 9, "g_first_norm": 221.3087921142578, "vocab_size": 50257, "entropy": 0.9145693778991699, "entropy_per_token": [0.6549602150917053, 0.9732266664505005, 0.1650102138519287, 1.8546264171600342, 1.0247783660888672, 0.4949021339416504, 0.020109618082642555, 0.8445966243743896, 0.0005976200336590409, 1.1704062223434448, 0.328832745552063, 0.667791485786438, 2.4011640548706055, 1.5351448059082031, 0.4475276470184326, 0.08026938140392303, 1.123635172843933, 0.004444883204996586, 3.3139703273773193, 1.1853935718536377], "max_p": 0.6898070573806763, "max_p_per_token": [0.8475388288497925, 0.7329766154289246, 0.9657491445541382, 0.28451499342918396, 0.516258955001831, 0.8567285537719727, 0.9973280429840088, 0.7170541286468506, 0.9999532699584961, 0.44844746589660645, 0.9338474869728088, 0.6692571640014648, 0.2828767001628876, 0.5226764678955078, 0.8796619772911072, 0.986310601234436, 0.5705885887145996, 0.9994851350784302, 0.10605620592832565, 0.4788307845592499], "n_positions_probed": 1, "per_restart_best": [9.124794006347656]}
+{"step": 138, "discrete_loss": 10.400144577026367, "best_sample_loss": 8.984808921813965, "soft_loss": 7.460580348968506, "best_discrete": 8.984808921813965, "best_soft": 6.887118339538574, "best_argmax": 9.729219436645508, "best_sampling": 8.984808921813965, "relax_gap": 0.2826464772952558, "n_match": 8, "g_first_norm": 267.78900146484375, "vocab_size": 50257, "entropy": 0.885380208492279, "entropy_per_token": [0.6866341829299927, 0.9562727212905884, 0.1518504023551941, 1.8992592096328735, 1.0099148750305176, 0.477169394493103, 0.020491838455200195, 0.8258047103881836, 0.0005910850595682859, 1.308174729347229, 0.3277955651283264, 0.6967620253562927, 2.5654778480529785, 1.50270414352417, 0.5343248844146729, 0.09624572843313217, 0.9874870181083679, 0.0042044613510370255, 2.4819350242614746, 1.1745045185089111], "max_p": 0.7061290740966797, "max_p_per_token": [0.8368848562240601, 0.7396265268325806, 0.969273567199707, 0.2787571847438812, 0.526935338973999, 0.863292932510376, 0.9972741007804871, 0.7288527488708496, 0.9999538660049438, 0.4107471704483032, 0.9340587854385376, 0.6250930428504944, 0.23358888924121857, 0.54500412940979, 0.8386905193328857, 0.9828674793243408, 0.667961061000824, 0.9995167255401611, 0.4583204388618469, 0.48588207364082336], "n_positions_probed": 1, "per_restart_best": [8.984808921813965]}
+{"step": 139, "discrete_loss": 10.290913581848145, "best_sample_loss": 8.847575187683105, "soft_loss": 8.623153686523438, "best_discrete": 8.847575187683105, "best_soft": 6.887118339538574, "best_argmax": 9.729219436645508, "best_sampling": 8.847575187683105, "relax_gap": 0.1620614032039315, "n_match": 8, "g_first_norm": 172.76473999023438, "vocab_size": 50257, "entropy": 0.8634563684463501, "entropy_per_token": [0.7012301683425903, 1.0053884983062744, 0.1596921682357788, 1.8973536491394043, 1.009910225868225, 0.4782261550426483, 0.021210692822933197, 0.8206481337547302, 0.0005895023932680488, 1.2620537281036377, 0.34559932351112366, 0.6918849349021912, 2.4428601264953613, 1.50032639503479, 0.6090661883354187, 0.0965086817741394, 1.088541030883789, 0.004137856885790825, 3.106342315673828, 0.027556292712688446], "max_p": 0.7157753109931946, "max_p_per_token": [0.8312788009643555, 0.7212235927581787, 0.9671773910522461, 0.3027374744415283, 0.5191144943237305, 0.862827479839325, 0.9971597194671631, 0.7317890524864197, 0.9999539852142334, 0.41486066579818726, 0.9293044209480286, 0.6257727146148682, 0.30755290389060974, 0.5455538630485535, 0.793419599533081, 0.9828277826309204, 0.6193187236785889, 0.9995256662368774, 0.1677834838628769, 0.9963243007659912], "n_positions_probed": 1, "per_restart_best": [8.847575187683105]}
+{"step": 140, "discrete_loss": 10.290913581848145, "best_sample_loss": 8.729960441589355, "soft_loss": 7.9197516441345215, "best_discrete": 8.729960441589355, "best_soft": 6.887118339538574, "best_argmax": 9.729219436645508, "best_sampling": 8.729960441589355, "relax_gap": 0.2304131619466759, "n_match": 7, "g_first_norm": 255.57498168945312, "vocab_size": 50257, "entropy": 0.8724725842475891, "entropy_per_token": [0.6506189703941345, 0.9616971611976624, 0.15864577889442444, 1.9208879470825195, 1.0005674362182617, 0.45394042134284973, 0.020885683596134186, 0.7954620122909546, 0.0005802656523883343, 1.2605834007263184, 0.3536413311958313, 0.671501874923706, 2.534203052520752, 1.479805588722229, 0.7043907642364502, 0.11559095978736877, 1.1165021657943726, 0.0040731108747422695, 3.2183871269226074, 0.027486801147460938], "max_p": 0.7111186385154724, "max_p_per_token": [0.8482602834701538, 0.7377733588218689, 0.967482328414917, 0.30747297406196594, 0.4986751675605774, 0.8725478649139404, 0.9972225427627563, 0.7453016638755798, 0.9999548196792603, 0.4245353043079376, 0.9266021251678467, 0.6544036865234375, 0.26388460397720337, 0.5565053224563599, 0.7237293720245361, 0.9785394668579102, 0.6047676205635071, 0.9995343685150146, 0.11888153105974197, 0.9962969422340393], "n_positions_probed": 1, "per_restart_best": [8.729960441589355]}
+{"step": 141, "discrete_loss": 10.452387809753418, "best_sample_loss": 8.758365631103516, "soft_loss": 7.598580837249756, "best_discrete": 8.729960441589355, "best_soft": 6.887118339538574, "best_argmax": 9.729219436645508, "best_sampling": 8.729960441589355, "relax_gap": 0.27302918954467936, "n_match": 6, "g_first_norm": 265.0986328125, "vocab_size": 50257, "entropy": 0.8831667304039001, "entropy_per_token": [0.6194837093353271, 1.1151888370513916, 0.15405844151973724, 1.951042890548706, 0.9916160106658936, 0.4307142496109009, 0.020663302391767502, 0.7732905745506287, 0.0005629650549963117, 1.2762471437454224, 0.35453543066978455, 0.6301407217979431, 2.585773468017578, 1.4543800354003906, 0.8090558648109436, 0.13762369751930237, 1.141779899597168, 0.0038798032328486443, 3.1865153312683105, 0.0267812367528677], "max_p": 0.6912807822227478, "max_p_per_token": [0.858026385307312, 0.44362878799438477, 0.9687363505363464, 0.3083476424217224, 0.47108951210975647, 0.8815218806266785, 0.9972706437110901, 0.7569937705993652, 0.9999563694000244, 0.4215921461582184, 0.9259911179542542, 0.7020571231842041, 0.2302100658416748, 0.5696557760238647, 0.608024537563324, 0.9732996821403503, 0.588447630405426, 0.9995595812797546, 0.124832883477211, 0.9963739514350891], "n_positions_probed": 1, "per_restart_best": [8.729960441589355]}
+{"step": 142, "discrete_loss": 9.546650886535645, "best_sample_loss": 8.661857604980469, "soft_loss": 7.506086826324463, "best_discrete": 8.661857604980469, "best_soft": 6.887118339538574, "best_argmax": 9.546650886535645, "best_sampling": 8.661857604980469, "relax_gap": 0.21374658866903173, "n_match": 7, "g_first_norm": 296.4619140625, "vocab_size": 50257, "entropy": 0.8740094304084778, "entropy_per_token": [0.6121412515640259, 1.0714792013168335, 0.1402956247329712, 1.9646731615066528, 0.9883764982223511, 0.3973243534564972, 0.020512394607067108, 0.750286340713501, 0.0005422792164608836, 1.2971971035003662, 0.33574625849723816, 0.7163254022598267, 2.5719192028045654, 1.4105167388916016, 0.8130597472190857, 0.17009520530700684, 1.0698999166488647, 0.003825646359473467, 3.1208314895629883, 0.025140345096588135], "max_p": 0.6944125294685364, "max_p_per_token": [0.8603944778442383, 0.5110639929771423, 0.972521185874939, 0.28922033309936523, 0.46171483397483826, 0.8941677808761597, 0.9973080158233643, 0.7685312628746033, 0.9999582767486572, 0.43217888474464417, 0.9312430024147034, 0.5526828169822693, 0.23171761631965637, 0.5917238593101501, 0.6442614793777466, 0.9651172161102295, 0.647790789604187, 0.9995666146278381, 0.140477254986763, 0.9966108202934265], "n_positions_probed": 1, "per_restart_best": [8.661857604980469]}
+{"step": 143, "discrete_loss": 9.614714622497559, "best_sample_loss": 8.578987121582031, "soft_loss": 6.8759636878967285, "best_discrete": 8.578987121582031, "best_soft": 6.8759636878967285, "best_argmax": 9.546650886535645, "best_sampling": 8.578987121582031, "relax_gap": 0.28484994533196045, "n_match": 7, "g_first_norm": 224.42559814453125, "vocab_size": 50257, "entropy": 0.8636911511421204, "entropy_per_token": [0.6429251432418823, 1.0576893091201782, 0.13626503944396973, 1.916388750076294, 0.9811446666717529, 0.3653174042701721, 0.020598936825990677, 0.7265095710754395, 0.0005121981957927346, 1.2826611995697021, 0.32398760318756104, 0.6570804119110107, 2.499725818634033, 1.3697713613510132, 0.78387451171875, 0.18276989459991455, 1.1332366466522217, 0.003728634212166071, 3.164748191833496, 0.024886969476938248], "max_p": 0.7047945857048035, "max_p_per_token": [0.850491464138031, 0.533343493938446, 0.9735421538352966, 0.33558234572410583, 0.4645112156867981, 0.9055902361869812, 0.9973058700561523, 0.7802878022193909, 0.9999607801437378, 0.4501309096813202, 0.9344860315322876, 0.672071635723114, 0.2384297251701355, 0.6113224625587463, 0.6814098954200745, 0.9617127776145935, 0.6041896939277649, 0.9995793700218201, 0.10530569404363632, 0.9966374635696411], "n_positions_probed": 1, "per_restart_best": [8.578987121582031]}
+{"step": 144, "discrete_loss": 9.614714622497559, "best_sample_loss": 9.095450401306152, "soft_loss": 6.729336738586426, "best_discrete": 8.578987121582031, "best_soft": 6.729336738586426, "best_argmax": 9.546650886535645, "best_sampling": 8.578987121582031, "relax_gap": 0.3001002106874405, "n_match": 7, "g_first_norm": 218.27630615234375, "vocab_size": 50257, "entropy": 0.8654770255088806, "entropy_per_token": [0.6587180495262146, 1.0262733697891235, 0.1298450082540512, 1.9896537065505981, 0.9835208654403687, 0.34379827976226807, 0.020726369693875313, 0.7076280117034912, 0.0005098882829770446, 1.2611042261123657, 0.3156714141368866, 0.7005560398101807, 2.5228264331817627, 1.3264657258987427, 0.7550925016403198, 0.20357587933540344, 1.2085726261138916, 0.0036242841742932796, 3.1268129348754883, 0.024565059691667557], "max_p": 0.7035180330276489, "max_p_per_token": [0.8450401425361633, 0.5611475110054016, 0.9751717448234558, 0.3167745769023895, 0.46682459115982056, 0.9129414558410645, 0.9972963929176331, 0.7893621921539307, 0.9999608993530273, 0.47726356983184814, 0.9369261264801025, 0.5998168587684631, 0.23645009100437164, 0.6303324699401855, 0.7074447274208069, 0.9558718204498291, 0.565517008304596, 0.9995929598808289, 0.09994813054800034, 0.9966773986816406], "n_positions_probed": 1, "per_restart_best": [8.578987121582031]}
+{"step": 145, "discrete_loss": 9.614714622497559, "best_sample_loss": 8.702086448669434, "soft_loss": 6.6358842849731445, "best_discrete": 8.578987121582031, "best_soft": 6.6358842849731445, "best_argmax": 9.546650886535645, "best_sampling": 8.578987121582031, "relax_gap": 0.3098199431269881, "n_match": 7, "g_first_norm": 208.02557373046875, "vocab_size": 50257, "entropy": 0.8607511520385742, "entropy_per_token": [0.6696678400039673, 1.0163236856460571, 0.12731263041496277, 1.9859923124313354, 0.9779553413391113, 0.32304057478904724, 0.020559756085276604, 0.6897850632667542, 0.0005038519739173353, 1.2654188871383667, 0.30591773986816406, 0.6885528564453125, 2.4746785163879395, 1.3040540218353271, 0.7255297899246216, 0.21892070770263672, 1.2666211128234863, 0.003461467567831278, 3.126674175262451, 0.02405237779021263], "max_p": 0.7063154578208923, "max_p_per_token": [0.8406748175621033, 0.574917197227478, 0.9758108854293823, 0.3244735896587372, 0.4681921899318695, 0.9197490215301514, 0.9973341226577759, 0.7977949380874634, 0.9999613761901855, 0.4745190143585205, 0.9397046566009521, 0.6229207515716553, 0.2433682084083557, 0.6402351260185242, 0.73140549659729, 0.951400876045227, 0.5197911858558655, 0.9996138215065002, 0.10769104957580566, 0.996749997138977], "n_positions_probed": 1, "per_restart_best": [8.578987121582031]}
+{"step": 146, "discrete_loss": 9.614714622497559, "best_sample_loss": 8.55627727508545, "soft_loss": 6.573777675628662, "best_discrete": 8.55627727508545, "best_soft": 6.573777675628662, "best_argmax": 9.546650886535645, "best_sampling": 8.55627727508545, "relax_gap": 0.31627948059460653, "n_match": 6, "g_first_norm": 212.73033142089844, "vocab_size": 50257, "entropy": 0.8634169697761536, "entropy_per_token": [0.6619225740432739, 0.9993160367012024, 0.12432704120874405, 1.978090524673462, 0.9725898504257202, 0.30527281761169434, 0.020631618797779083, 0.6798056960105896, 0.0005036008078604937, 1.2627965211868286, 0.3048543930053711, 0.7125901579856873, 2.489224910736084, 1.2832540273666382, 0.6982495188713074, 0.23904745280742645, 1.3910081386566162, 0.003319690702483058, 3.117785930633545, 0.023748274892568588], "max_p": 0.7036052346229553, "max_p_per_token": [0.8419670462608337, 0.5883510708808899, 0.9765865802764893, 0.32713696360588074, 0.47604575753211975, 0.9254987835884094, 0.9973292350769043, 0.8024434447288513, 0.9999613761901855, 0.47867435216903687, 0.9402090311050415, 0.5677136778831482, 0.24199263751506805, 0.6486997604370117, 0.7503679394721985, 0.9453607201576233, 0.47182613611221313, 0.9996318817138672, 0.09551454335451126, 0.996793806552887], "n_positions_probed": 1, "per_restart_best": [8.55627727508545]}
+{"step": 147, "discrete_loss": 9.227023124694824, "best_sample_loss": 8.767717361450195, "soft_loss": 6.50726842880249, "best_discrete": 8.55627727508545, "best_soft": 6.50726842880249, "best_argmax": 9.227023124694824, "best_sampling": 8.55627727508545, "relax_gap": 0.29475971384674376, "n_match": 6, "g_first_norm": 277.9533386230469, "vocab_size": 50257, "entropy": 0.8600128293037415, "entropy_per_token": [0.6134162545204163, 1.0132577419281006, 0.12320555746555328, 2.001354932785034, 0.9717750549316406, 0.2956312298774719, 0.01999093033373356, 0.6763548851013184, 0.0005014491034671664, 1.2719193696975708, 0.3004741072654724, 0.7017191052436829, 2.430701732635498, 1.2847250699996948, 0.6762268543243408, 0.2468736320734024, 1.4448268413543701, 0.0032094502821564674, 3.1007657051086426, 0.02332637645304203], "max_p": 0.7093009948730469, "max_p_per_token": [0.856879711151123, 0.5827988386154175, 0.9769166111946106, 0.3110353648662567, 0.49130192399024963, 0.9285467863082886, 0.997437596321106, 0.8022187948226929, 0.9999614953994751, 0.471605122089386, 0.9415714740753174, 0.5991844534873962, 0.2660216987133026, 0.6494844555854797, 0.7662261128425598, 0.9430074095726013, 0.4867926239967346, 0.9996459484100342, 0.11852023005485535, 0.9968627691268921], "n_positions_probed": 1, "per_restart_best": [8.55627727508545]}
+{"step": 148, "discrete_loss": 9.930482864379883, "best_sample_loss": 8.552266120910645, "soft_loss": 6.635514259338379, "best_discrete": 8.552266120910645, "best_soft": 6.50726842880249, "best_argmax": 9.227023124694824, "best_sampling": 8.552266120910645, "relax_gap": 0.3318034631387747, "n_match": 6, "g_first_norm": 297.5418395996094, "vocab_size": 50257, "entropy": 0.8656999468803406, "entropy_per_token": [0.6791102290153503, 1.025608777999878, 0.12416303157806396, 1.9666919708251953, 0.9706937670707703, 0.2854178547859192, 0.019500069320201874, 0.6960826516151428, 0.0004987511201761663, 1.2673834562301636, 0.30491966009140015, 0.7179105281829834, 2.4592959880828857, 1.2744691371917725, 0.6620012521743774, 0.25447988510131836, 1.5177600383758545, 0.0028684011194854975, 3.061089038848877, 0.024055011570453644], "max_p": 0.6976301074028015, "max_p_per_token": [0.8329724073410034, 0.5724601745605469, 0.9767157435417175, 0.3198030889034271, 0.4968497157096863, 0.9317354559898376, 0.9975265860557556, 0.7915076613426208, 0.9999618530273438, 0.4743534028530121, 0.940605878829956, 0.5445654392242432, 0.22474049031734467, 0.6545288562774658, 0.7759467959403992, 0.9405145049095154, 0.3853040337562561, 0.9996883869171143, 0.09608776867389679, 0.9967347979545593], "n_positions_probed": 1, "per_restart_best": [8.552266120910645]}
+{"step": 149, "discrete_loss": 9.22107982635498, "best_sample_loss": 8.506309509277344, "soft_loss": 6.780149936676025, "best_discrete": 8.506309509277344, "best_soft": 6.50726842880249, "best_argmax": 9.22107982635498, "best_sampling": 8.506309509277344, "relax_gap": 0.2647119356566546, "n_match": 6, "g_first_norm": 277.78887939453125, "vocab_size": 50257, "entropy": 0.8427647948265076, "entropy_per_token": [0.5614901781082153, 1.0389844179153442, 0.12290629744529724, 1.9678212404251099, 0.9632745981216431, 0.2748876214027405, 0.01934613659977913, 0.685698390007019, 0.0005269752582535148, 0.9264821410179138, 0.31116005778312683, 0.70964515209198, 2.3686609268188477, 1.2560346126556396, 0.6513514518737793, 0.2792273163795471, 1.5892058610916138, 0.0029501542448997498, 3.102010488510132, 0.023632727563381195], "max_p": 0.7137467265129089, "max_p_per_token": [0.869940996170044, 0.5573208332061768, 0.9770548343658447, 0.3180125653743744, 0.5255489349365234, 0.9350463151931763, 0.9975619316101074, 0.7965360879898071, 0.9999595880508423, 0.682327389717102, 0.9389845132827759, 0.5726458430290222, 0.2624173164367676, 0.6614524722099304, 0.7814285159111023, 0.9323371052742004, 0.35376328229904175, 0.9996788501739502, 0.1161143034696579, 0.9968032836914062], "n_positions_probed": 1, "per_restart_best": [8.506309509277344]}
+{"step": 150, "discrete_loss": 8.888798713684082, "best_sample_loss": 8.499072074890137, "soft_loss": 6.4280290603637695, "best_discrete": 8.499072074890137, "best_soft": 6.4280290603637695, "best_argmax": 8.888798713684082, "best_sampling": 8.499072074890137, "relax_gap": 0.2768393944540582, "n_match": 5, "g_first_norm": 189.00601196289062, "vocab_size": 50257, "entropy": 0.8369820713996887, "entropy_per_token": [0.5820147395133972, 0.9750205874443054, 0.12445741891860962, 1.927802324295044, 0.9451410174369812, 0.27062568068504333, 0.01946604810655117, 0.6860998868942261, 0.0005012141773477197, 0.9404483437538147, 0.30253463983535767, 0.700728714466095, 2.346987724304199, 1.2503540515899658, 0.6411364078521729, 0.2859083414077759, 1.6015503406524658, 0.002870945492759347, 3.1117706298828125, 0.024222582578659058], "max_p": 0.7189220786094666, "max_p_per_token": [0.8637552857398987, 0.6273303031921387, 0.9767330288887024, 0.33411905169487, 0.5684816837310791, 0.9361128211021423, 0.9975558519363403, 0.7961674332618713, 0.9999617338180542, 0.6768952012062073, 0.9428672790527344, 0.5977897047996521, 0.23752950131893158, 0.6655254364013672, 0.7900129556655884, 0.9301702380180359, 0.32525238394737244, 0.9996888637542725, 0.11579153686761856, 0.996699869632721], "n_positions_probed": 1, "per_restart_best": [8.499072074890137]}
+{"step": 151, "discrete_loss": 8.888798713684082, "best_sample_loss": 8.674701690673828, "soft_loss": 6.256644248962402, "best_discrete": 8.499072074890137, "best_soft": 6.256644248962402, "best_argmax": 8.888798713684082, "best_sampling": 8.499072074890137, "relax_gap": 0.29612038133674284, "n_match": 5, "g_first_norm": 162.94027709960938, "vocab_size": 50257, "entropy": 0.8009632229804993, "entropy_per_token": [0.5849224328994751, 0.9304673075675964, 0.1245887279510498, 1.9097514152526855, 0.9203497171401978, 0.26926472783088684, 0.019981812685728073, 0.6888531446456909, 0.00048799975775182247, 0.9507189393043518, 0.2923308312892914, 0.05115246772766113, 2.352058172225952, 1.2457826137542725, 0.6289631128311157, 0.29284006357192993, 1.5931485891342163, 0.0027847723104059696, 3.13607120513916, 0.024746384471654892], "max_p": 0.7409282326698303, "max_p_per_token": [0.8625530004501343, 0.6644454002380371, 0.9767560958862305, 0.3365902900695801, 0.6029723286628723, 0.9363280534744263, 0.997490644454956, 0.7948707342147827, 0.9999628067016602, 0.6747463941574097, 0.9456154704093933, 0.9911831617355347, 0.22723151743412018, 0.6681734323501587, 0.7989125847816467, 0.927802324295044, 0.31677818298339844, 0.9996998310089111, 0.09984124451875687, 0.9966110587120056], "n_positions_probed": 1, "per_restart_best": [8.499072074890137]}
+{"step": 152, "discrete_loss": 9.933547019958496, "best_sample_loss": 8.21744155883789, "soft_loss": 6.4051055908203125, "best_discrete": 8.21744155883789, "best_soft": 6.256644248962402, "best_argmax": 8.888798713684082, "best_sampling": 8.21744155883789, "relax_gap": 0.355204583221767, "n_match": 5, "g_first_norm": 176.79823303222656, "vocab_size": 50257, "entropy": 0.7342970967292786, "entropy_per_token": [0.5332788825035095, 0.8204417824745178, 0.12039601802825928, 1.908857822418213, 0.8938812017440796, 0.27283522486686707, 0.020413102582097054, 0.6957265138626099, 0.00048705178778618574, 0.9568919539451599, 0.2895122468471527, 0.05712934955954552, 1.3016257286071777, 1.2180416584014893, 0.6174036860466003, 0.3028992712497711, 1.5810585021972656, 0.0026639758143574, 3.0668141841888428, 0.02558353915810585], "max_p": 0.7715075016021729, "max_p_per_token": [0.8786818981170654, 0.7328208684921265, 0.9778301119804382, 0.32970303297042847, 0.6330263614654541, 0.9350212812423706, 0.9974397420883179, 0.7913920283317566, 0.9999628067016602, 0.6755052208900452, 0.9465126991271973, 0.989912211894989, 0.7038990259170532, 0.679722785949707, 0.8063886761665344, 0.9245445132255554, 0.30441904067993164, 0.9997147917747498, 0.1271829903125763, 0.9964699745178223], "n_positions_probed": 1, "per_restart_best": [8.21744155883789]}
+{"step": 153, "discrete_loss": 8.774279594421387, "best_sample_loss": 9.121261596679688, "soft_loss": 6.676928520202637, "best_discrete": 8.21744155883789, "best_soft": 6.256644248962402, "best_argmax": 8.774279594421387, "best_sampling": 8.21744155883789, "relax_gap": 0.2390339915258944, "n_match": 5, "g_first_norm": 185.01019287109375, "vocab_size": 50257, "entropy": 0.7375218272209167, "entropy_per_token": [0.5443528294563293, 0.7133010625839233, 0.11487343907356262, 1.9378621578216553, 0.8975893259048462, 0.277180552482605, 0.020805723965168, 0.7003146409988403, 0.00047872232971712947, 0.9711411595344543, 0.2902541160583496, 0.06395966559648514, 1.3639471530914307, 1.2833638191223145, 0.6161742210388184, 0.3086984157562256, 1.5526161193847656, 0.0026362412609159946, 3.064639091491699, 0.026248207315802574], "max_p": 0.7710424661636353, "max_p_per_token": [0.8762171864509583, 0.7921936511993408, 0.9791138768196106, 0.31374382972717285, 0.6342636346817017, 0.9335960745811462, 0.9973937273025513, 0.7892361283302307, 0.9999635219573975, 0.6729052662849426, 0.9464735388755798, 0.9884124398231506, 0.6788856983184814, 0.662106990814209, 0.8068237900733948, 0.922945499420166, 0.30787262320518494, 0.9997187256813049, 0.12262801826000214, 0.9963539838790894], "n_positions_probed": 1, "per_restart_best": [8.21744155883789]}
+{"step": 154, "discrete_loss": 9.933547019958496, "best_sample_loss": 9.277132987976074, "soft_loss": 6.595285892486572, "best_discrete": 8.21744155883789, "best_soft": 6.256644248962402, "best_argmax": 8.774279594421387, "best_sampling": 8.21744155883789, "relax_gap": 0.33605932712299896, "n_match": 5, "g_first_norm": 137.21360778808594, "vocab_size": 50257, "entropy": 0.7412042021751404, "entropy_per_token": [0.5308350324630737, 0.6769037842750549, 0.11054578423500061, 1.9419527053833008, 0.8845937848091125, 0.27933183312416077, 0.021300839260220528, 0.7000893354415894, 0.0004740412114188075, 0.9873407483100891, 0.2889971435070038, 0.07229413092136383, 1.3936494588851929, 1.2485973834991455, 0.7376803159713745, 0.3199978768825531, 1.5366261005401611, 0.0025800217408686876, 3.063039779663086, 0.02725527249276638], "max_p": 0.7709981799125671, "max_p_per_token": [0.8802756667137146, 0.8100119829177856, 0.9801346659660339, 0.3062300980091095, 0.6484586596488953, 0.9328104853630066, 0.9973305463790894, 0.7891103625297546, 0.9999639987945557, 0.6680334210395813, 0.9469400644302368, 0.9865208864212036, 0.6640110611915588, 0.6753129363059998, 0.787852942943573, 0.9193091988563538, 0.3052031993865967, 0.9997259974479675, 0.12654584646224976, 0.9961810111999512], "n_positions_probed": 1, "per_restart_best": [8.21744155883789]}
+{"step": 155, "discrete_loss": 9.915837287902832, "best_sample_loss": 8.174875259399414, "soft_loss": 6.546984672546387, "best_discrete": 8.174875259399414, "best_soft": 6.256644248962402, "best_argmax": 8.774279594421387, "best_sampling": 8.174875259399414, "relax_gap": 0.3397446446067034, "n_match": 4, "g_first_norm": 139.16612243652344, "vocab_size": 50257, "entropy": 0.753501832485199, "entropy_per_token": [0.5176223516464233, 0.6384508609771729, 0.10642971098423004, 1.94743013381958, 0.8701821565628052, 0.28172022104263306, 0.021776704117655754, 0.7038986086845398, 0.0004716843832284212, 1.0059806108474731, 0.2884889841079712, 0.08179827034473419, 1.422197937965393, 1.2193927764892578, 0.724738359451294, 0.6254832148551941, 1.516782283782959, 0.002550216391682625, 3.066404342651367, 0.028237231075763702], "max_p": 0.7604753971099854, "max_p_per_token": [0.8844250440597534, 0.8276271820068359, 0.9810804724693298, 0.29785940051078796, 0.6623409986495972, 0.9319534301757812, 0.997270405292511, 0.7871970534324646, 0.9999641180038452, 0.661430299282074, 0.947196364402771, 0.9842851758003235, 0.6486791968345642, 0.6860737204551697, 0.7931643724441528, 0.7064924240112305, 0.29831749200820923, 0.9997301697731018, 0.11840888112783432, 0.99601149559021], "n_positions_probed": 1, "per_restart_best": [8.174875259399414]}
+{"step": 156, "discrete_loss": 9.035216331481934, "best_sample_loss": 8.428332328796387, "soft_loss": 7.029871940612793, "best_discrete": 8.174875259399414, "best_soft": 6.256644248962402, "best_argmax": 8.774279594421387, "best_sampling": 8.174875259399414, "relax_gap": 0.22194757903934206, "n_match": 4, "g_first_norm": 253.3656463623047, "vocab_size": 50257, "entropy": 0.7384964823722839, "entropy_per_token": [0.5192051529884338, 0.5770381689071655, 0.10121479630470276, 1.9782917499542236, 0.8275665044784546, 0.2845171093940735, 0.022890053689479828, 0.7167252898216248, 0.00046062376350164413, 1.0271145105361938, 0.29805028438568115, 0.08519989252090454, 1.4947575330734253, 1.1807262897491455, 0.7055633664131165, 0.7141368389129639, 1.1491972208023071, 0.0026958677917718887, 3.0553550720214844, 0.029223579913377762], "max_p": 0.7658001184463501, "max_p_per_token": [0.8850559592247009, 0.8538566827774048, 0.9822397232055664, 0.2852710485458374, 0.695010244846344, 0.9308630228042603, 0.9971141815185547, 0.7813258171081543, 0.9999650716781616, 0.6541725993156433, 0.9449678659439087, 0.9834733605384827, 0.614859402179718, 0.7004601955413818, 0.8012681007385254, 0.5076911449432373, 0.61359703540802, 0.9997139573097229, 0.08926498144865036, 0.9958310723304749], "n_positions_probed": 1, "per_restart_best": [8.174875259399414]}
+{"step": 157, "discrete_loss": 8.774279594421387, "best_sample_loss": 8.468680381774902, "soft_loss": 6.734711647033691, "best_discrete": 8.174875259399414, "best_soft": 6.256644248962402, "best_argmax": 8.774279594421387, "best_sampling": 8.174875259399414, "relax_gap": 0.2324484791531416, "n_match": 4, "g_first_norm": 194.1800537109375, "vocab_size": 50257, "entropy": 0.7427008748054504, "entropy_per_token": [0.5326525568962097, 0.5887278318405151, 0.09655977785587311, 2.0095009803771973, 0.8778609037399292, 0.28305891156196594, 0.0234590545296669, 0.7098550796508789, 0.00045287946704775095, 1.0519345998764038, 0.3041967749595642, 0.0945606455206871, 1.5044764280319214, 1.1519712209701538, 0.686732292175293, 0.7085301876068115, 1.1956086158752441, 0.0024622732307761908, 3.000535011291504, 0.030881524085998535], "max_p": 0.7662423849105835, "max_p_per_token": [0.8808040618896484, 0.8500117063522339, 0.983281672000885, 0.27100345492362976, 0.662988007068634, 0.9311171174049377, 0.9970340728759766, 0.7844063639640808, 0.9999655485153198, 0.6424932479858398, 0.9436543583869934, 0.9811736345291138, 0.6033372282981873, 0.7106723189353943, 0.81095290184021, 0.5557267069816589, 0.5831832885742188, 0.9997425675392151, 0.13776761293411255, 0.9955310225486755], "n_positions_probed": 1, "per_restart_best": [8.174875259399414]}
+{"step": 158, "discrete_loss": 8.898820877075195, "best_sample_loss": 8.151519775390625, "soft_loss": 6.514739036560059, "best_discrete": 8.151519775390625, "best_soft": 6.256644248962402, "best_argmax": 8.774279594421387, "best_sampling": 8.151519775390625, "relax_gap": 0.26790985833380665, "n_match": 4, "g_first_norm": 147.25704956054688, "vocab_size": 50257, "entropy": 0.6828792691230774, "entropy_per_token": [0.5329858660697937, 0.5894368290901184, 0.09304672479629517, 2.0197365283966064, 0.8814230561256409, 0.2827662229537964, 0.0239473395049572, 0.712182879447937, 0.00045393023174256086, 1.081168293952942, 0.31596803665161133, 0.10544445365667343, 1.5261034965515137, 1.134418249130249, 0.6715890169143677, 0.707430362701416, 1.2334766387939453, 0.0023291180841624737, 1.7112401723861694, 0.03243740275502205], "max_p": 0.7889288067817688, "max_p_per_token": [0.8806868195533752, 0.8501707911491394, 0.9840565919876099, 0.2629961669445038, 0.6614976525306702, 0.9310712814331055, 0.9969664216041565, 0.7830464839935303, 0.9999656677246094, 0.6277580857276917, 0.9410053491592407, 0.9784032106399536, 0.5853428244590759, 0.716640830039978, 0.8180757164955139, 0.5679884552955627, 0.5584535002708435, 0.9997592568397522, 0.6394413113594055, 0.99524986743927], "n_positions_probed": 1, "per_restart_best": [8.151519775390625]}
+{"step": 159, "discrete_loss": 8.898820877075195, "best_sample_loss": 8.151519775390625, "soft_loss": 8.280113220214844, "best_discrete": 8.151519775390625, "best_soft": 6.256644248962402, "best_argmax": 8.774279594421387, "best_sampling": 8.151519775390625, "relax_gap": 0.06952692557889807, "n_match": 4, "g_first_norm": 173.4971160888672, "vocab_size": 50257, "entropy": 0.7236052751541138, "entropy_per_token": [0.5593172907829285, 0.5625611543655396, 0.08980671316385269, 2.0814998149871826, 0.9332335591316223, 0.2839309275150299, 0.023410703986883163, 0.7179068326950073, 0.0004635975928977132, 1.1423194408416748, 0.3221900761127472, 0.11360197514295578, 1.6399805545806885, 1.109405279159546, 0.6947685480117798, 0.7183950543403625, 1.3298430442810059, 0.00235367170535028, 2.1471023559570312, 1.3886471606383566e-05], "max_p": 0.7681896090507507, "max_p_per_token": [0.8728536367416382, 0.8599504828453064, 0.9847442507743835, 0.24701102077960968, 0.6214670538902283, 0.9307352304458618, 0.9970491528511047, 0.7789499163627625, 0.999964714050293, 0.590729832649231, 0.9396808743476868, 0.9762566089630127, 0.513124942779541, 0.7235634326934814, 0.8096854090690613, 0.524911105632782, 0.48503217101097107, 0.999757707118988, 0.5083261728286743, 0.9999991655349731], "n_positions_probed": 1, "per_restart_best": [8.151519775390625]}
+{"step": 160, "discrete_loss": 8.898820877075195, "best_sample_loss": 8.099212646484375, "soft_loss": 7.875463962554932, "best_discrete": 8.099212646484375, "best_soft": 6.256644248962402, "best_argmax": 8.774279594421387, "best_sampling": 8.099212646484375, "relax_gap": 0.1149991587263653, "n_match": 4, "g_first_norm": 130.52024841308594, "vocab_size": 50257, "entropy": 0.7810370922088623, "entropy_per_token": [0.9726859331130981, 0.5492862462997437, 0.08756177127361298, 2.105868101119995, 0.9527587890625, 0.28627756237983704, 0.023627419024705887, 0.7238903045654297, 0.00046281161485239863, 1.2071491479873657, 0.3385940194129944, 0.12285936623811722, 1.719288945198059, 1.0778392553329468, 0.7042189836502075, 0.7219751477241516, 1.3663231134414673, 0.0023867397103458643, 2.657672643661499, 1.539101685921196e-05], "max_p": 0.7400755882263184, "max_p_per_token": [0.6540761590003967, 0.8649154305458069, 0.9852096438407898, 0.24670690298080444, 0.6035844087600708, 0.9299670457839966, 0.9970167875289917, 0.7749336361885071, 0.9999649524688721, 0.5496481657028198, 0.9360755085945129, 0.973766028881073, 0.46104469895362854, 0.7339641451835632, 0.8071506023406982, 0.5109577775001526, 0.4655994176864624, 0.999755322933197, 0.3071770668029785, 0.9999990463256836], "n_positions_probed": 1, "per_restart_best": [8.099212646484375]}
+{"step": 161, "discrete_loss": 8.709681510925293, "best_sample_loss": 8.19849967956543, "soft_loss": 7.282945156097412, "best_discrete": 8.099212646484375, "best_soft": 6.256644248962402, "best_argmax": 8.709681510925293, "best_sampling": 8.099212646484375, "relax_gap": 0.16381039341544285, "n_match": 4, "g_first_norm": 157.31130981445312, "vocab_size": 50257, "entropy": 0.7847484946250916, "entropy_per_token": [0.945163905620575, 0.5161520838737488, 0.0856696143746376, 2.1043272018432617, 0.947944164276123, 0.2861137390136719, 0.02445293590426445, 0.7301957011222839, 0.00046068569645285606, 1.1711560487747192, 0.358089804649353, 0.13622525334358215, 1.7488619089126587, 1.0486886501312256, 0.6942565441131592, 0.7216047048568726, 1.333893060684204, 0.002490327460691333, 2.839205741882324, 1.6440961189800873e-05], "max_p": 0.7386927008628845, "max_p_per_token": [0.6696817278862, 0.8746245503425598, 0.9855959415435791, 0.25982415676116943, 0.6064524054527283, 0.9299479722976685, 0.9968966245651245, 0.7708932757377625, 0.9999650716781616, 0.5767306685447693, 0.9315513968467712, 0.9700571298599243, 0.44131481647491455, 0.7439405918121338, 0.8127448558807373, 0.533709704875946, 0.48798054456710815, 0.9997450709342957, 0.1821974217891693, 0.999998927116394], "n_positions_probed": 1, "per_restart_best": [8.099212646484375]}
+{"step": 162, "discrete_loss": 8.709681510925293, "best_sample_loss": 8.076803207397461, "soft_loss": 6.536416530609131, "best_discrete": 8.076803207397461, "best_soft": 6.256644248962402, "best_argmax": 8.709681510925293, "best_sampling": 8.076803207397461, "relax_gap": 0.24952289903942543, "n_match": 4, "g_first_norm": 148.2299041748047, "vocab_size": 50257, "entropy": 0.792191743850708, "entropy_per_token": [0.9124204516410828, 0.5278281569480896, 0.09995466470718384, 2.085127830505371, 0.9295238256454468, 0.2880965769290924, 0.02506888285279274, 0.7340877652168274, 0.0004611497570294887, 1.2341729402542114, 0.3692234754562378, 0.14791372418403625, 1.7841219902038574, 1.0370292663574219, 0.66761714220047, 0.7194967269897461, 1.3198392391204834, 0.0024726458359509706, 2.9593605995178223, 1.756454184942413e-05], "max_p": 0.7378792762756348, "max_p_per_token": [0.6926186680793762, 0.8708009123802185, 0.983626663684845, 0.27548494935035706, 0.6231307983398438, 0.9291384816169739, 0.996807873249054, 0.7685709595680237, 0.9999650716781616, 0.5380402207374573, 0.928952157497406, 0.9666929841041565, 0.4118356704711914, 0.7477718591690063, 0.8237941861152649, 0.5541760921478271, 0.503490149974823, 0.9997485280036926, 0.142940953373909, 0.999998927116394], "n_positions_probed": 1, "per_restart_best": [8.076803207397461]}
+{"step": 163, "discrete_loss": 8.709681510925293, "best_sample_loss": 8.049789428710938, "soft_loss": 6.424338340759277, "best_discrete": 8.049789428710938, "best_soft": 6.256644248962402, "best_argmax": 8.709681510925293, "best_sampling": 8.049789428710938, "relax_gap": 0.26239112960666994, "n_match": 4, "g_first_norm": 149.00184631347656, "vocab_size": 50257, "entropy": 0.7971785068511963, "entropy_per_token": [0.8723411560058594, 0.5419480204582214, 0.0976056307554245, 2.220313310623169, 0.9074956774711609, 0.28867873549461365, 0.025786172598600388, 0.7387970089912415, 0.00046623655362054706, 1.180226445198059, 0.37939149141311646, 0.1616860032081604, 1.8575105667114258, 1.023728370666504, 0.6449487209320068, 0.7234338521957397, 1.3103580474853516, 0.0024399026297032833, 2.966395854949951, 1.8667440599529073e-05], "max_p": 0.738389790058136, "max_p_per_token": [0.7163365483283997, 0.8660109639167786, 0.9841145277023315, 0.2552369236946106, 0.6406775116920471, 0.928794264793396, 0.9967045187950134, 0.7658405303955078, 0.9999645948410034, 0.5749399065971375, 0.9265558123588562, 0.9625956416130066, 0.36304348707199097, 0.7521005868911743, 0.8330101370811462, 0.5478241443634033, 0.5120893716812134, 0.9997536540031433, 0.14220355451107025, 0.9999988079071045], "n_positions_probed": 1, "per_restart_best": [8.049789428710938]}
+{"step": 164, "discrete_loss": 8.709681510925293, "best_sample_loss": 8.131245613098145, "soft_loss": 6.346612453460693, "best_discrete": 8.049789428710938, "best_soft": 6.256644248962402, "best_argmax": 8.709681510925293, "best_sampling": 8.049789428710938, "relax_gap": 0.27131520877088344, "n_match": 4, "g_first_norm": 148.87991333007812, "vocab_size": 50257, "entropy": 0.8051458597183228, "entropy_per_token": [0.8385190963745117, 0.5604729652404785, 0.09633992612361908, 2.191669464111328, 0.9342214465141296, 0.29141348600387573, 0.02628019079566002, 0.7460907697677612, 0.00047140236711129546, 1.2119600772857666, 0.3854854702949524, 0.17178437113761902, 1.974948763847351, 1.0214817523956299, 0.6293380260467529, 0.7274757623672485, 1.2937928438186646, 0.0023983949795365334, 2.9987540245056152, 1.9651146430987865e-05], "max_p": 0.7296944856643677, "max_p_per_token": [0.7363739609718323, 0.8598276376724243, 0.9843990802764893, 0.27410805225372314, 0.5305492281913757, 0.9276990294456482, 0.9966339468955994, 0.761773943901062, 0.9999642372131348, 0.5564116835594177, 0.9252074360847473, 0.9595032930374146, 0.29431799054145813, 0.752663791179657, 0.8388700485229492, 0.5420834422111511, 0.5235434174537659, 0.9997597336769104, 0.13020016252994537, 0.9999986886978149], "n_positions_probed": 1, "per_restart_best": [8.049789428710938]}
+{"step": 165, "discrete_loss": 8.675838470458984, "best_sample_loss": 8.008723258972168, "soft_loss": 6.3397321701049805, "best_discrete": 8.008723258972168, "best_soft": 6.256644248962402, "best_argmax": 8.675838470458984, "best_sampling": 8.008723258972168, "relax_gap": 0.2692657670274047, "n_match": 4, "g_first_norm": 155.16851806640625, "vocab_size": 50257, "entropy": 0.8195711374282837, "entropy_per_token": [0.7878326177597046, 0.5821073651313782, 0.09539534151554108, 2.156553268432617, 0.9174841046333313, 0.6543828845024109, 0.026555150747299194, 0.7544248104095459, 0.0004790955572389066, 1.1529943943023682, 0.3910464644432068, 0.18225525319576263, 1.9972665309906006, 1.024664282798767, 0.6160216331481934, 0.7332638502120972, 1.3120417594909668, 0.002345642074942589, 3.0042884349823, 2.0621037037926726e-05], "max_p": 0.7193182706832886, "max_p_per_token": [0.7625478506088257, 0.8523344397544861, 0.9846187233924866, 0.2933602035045624, 0.5699601769447327, 0.6685963273048401, 0.9965952038764954, 0.7567522525787354, 0.9999635219573975, 0.5939677953720093, 0.9240521192550659, 0.95621657371521, 0.2707662284374237, 0.7513477206230164, 0.8438815474510193, 0.5257464647293091, 0.5066578984260559, 0.999767005443573, 0.1292344033718109, 0.9999986886978149], "n_positions_probed": 1, "per_restart_best": [8.008723258972168]}
+{"step": 166, "discrete_loss": 8.235953330993652, "best_sample_loss": 7.963543891906738, "soft_loss": 6.315230846405029, "best_discrete": 7.963543891906738, "best_soft": 6.256644248962402, "best_argmax": 8.235953330993652, "best_sampling": 7.963543891906738, "relax_gap": 0.23321191942170602, "n_match": 4, "g_first_norm": 174.6647186279297, "vocab_size": 50257, "entropy": 0.8184637427330017, "entropy_per_token": [0.7200419902801514, 0.6086768507957458, 0.09616036713123322, 2.094768524169922, 0.9019711017608643, 0.6031925678253174, 0.027038797736167908, 0.7666661143302917, 0.0004787587677128613, 1.207434058189392, 0.395670622587204, 0.1925470381975174, 2.042318820953369, 1.0356475114822388, 0.6075425148010254, 0.7348042726516724, 1.2825706005096436, 0.0023201415315270424, 3.049403429031372, 2.160581607313361e-05], "max_p": 0.7235124111175537, "max_p_per_token": [0.792827844619751, 0.8431714177131653, 0.9845091700553894, 0.32074248790740967, 0.586267352104187, 0.7225703597068787, 0.9965548515319824, 0.7494977116584778, 0.999963641166687, 0.5611897706985474, 0.9231210947036743, 0.9529052376747131, 0.2576647996902466, 0.7474215626716614, 0.8468598127365112, 0.5401117205619812, 0.5254492163658142, 0.9997711777687073, 0.11965083330869675, 0.9999985694885254], "n_positions_probed": 1, "per_restart_best": [7.963543891906738]}
+{"step": 167, "discrete_loss": 8.235953330993652, "best_sample_loss": 8.02326488494873, "soft_loss": 6.209174156188965, "best_discrete": 7.963543891906738, "best_soft": 6.209174156188965, "best_argmax": 8.235953330993652, "best_sampling": 7.963543891906738, "relax_gap": 0.2460892010130126, "n_match": 4, "g_first_norm": 230.4794158935547, "vocab_size": 50257, "entropy": 0.8065705299377441, "entropy_per_token": [0.627362847328186, 0.6363213062286377, 0.09663517773151398, 2.044191360473633, 0.890418529510498, 0.4664962887763977, 0.02698509953916073, 0.7644513845443726, 0.00048451771726831794, 1.1432693004608154, 0.4039527177810669, 0.20495404303073883, 2.069611072540283, 1.0477252006530762, 0.5978097915649414, 0.7410507202148438, 1.3165900707244873, 0.002291284501552582, 3.0507864952087402, 2.2338710550684482e-05], "max_p": 0.7314451336860657, "max_p_per_token": [0.8305104970932007, 0.8332273364067078, 0.9844509363174438, 0.34412500262260437, 0.5961716175079346, 0.8267208933830261, 0.9965693950653076, 0.7446065545082092, 0.9999630451202393, 0.6018717288970947, 0.9212889075279236, 0.9488003849983215, 0.26453712582588196, 0.7429973483085632, 0.8505590558052063, 0.5265317559242249, 0.4957391321659088, 0.9997755885124207, 0.12045818567276001, 0.9999985694885254], "n_positions_probed": 1, "per_restart_best": [7.963543891906738]}
+{"step": 168, "discrete_loss": 8.235953330993652, "best_sample_loss": 7.978593349456787, "soft_loss": 6.151832580566406, "best_discrete": 7.963543891906738, "best_soft": 6.151832580566406, "best_argmax": 8.235953330993652, "best_sampling": 7.963543891906738, "relax_gap": 0.2530515493068974, "n_match": 4, "g_first_norm": 150.19232177734375, "vocab_size": 50257, "entropy": 0.8122369647026062, "entropy_per_token": [0.6036129593849182, 0.6559041142463684, 0.09488413482904434, 2.0089402198791504, 0.8859206438064575, 0.46883875131607056, 0.02709934674203396, 0.77670818567276, 0.0004868065007030964, 1.1795916557312012, 0.41031020879745483, 0.217292919754982, 2.120950698852539, 1.0581457614898682, 0.5881655216217041, 0.7445221543312073, 1.3073339462280273, 0.0023090264294296503, 3.0936996936798096, 2.3317748855333775e-05], "max_p": 0.7307536602020264, "max_p_per_token": [0.8404370546340942, 0.8264214396476746, 0.9848284125328064, 0.3601199984550476, 0.6021804809570312, 0.8251878619194031, 0.9965587258338928, 0.7366106510162354, 0.9999630451202393, 0.5801729559898376, 0.9199045896530151, 0.9446021914482117, 0.2589546740055084, 0.7392796277999878, 0.8539549708366394, 0.5358105301856995, 0.5025127530097961, 0.9997747540473938, 0.10780002176761627, 0.9999984502792358], "n_positions_probed": 1, "per_restart_best": [7.963543891906738]}
+{"step": 169, "discrete_loss": 8.235953330993652, "best_sample_loss": 7.9451165199279785, "soft_loss": 6.118430137634277, "best_discrete": 7.9451165199279785, "best_soft": 6.118430137634277, "best_argmax": 8.235953330993652, "best_sampling": 7.9451165199279785, "relax_gap": 0.25710723558749204, "n_match": 4, "g_first_norm": 154.18984985351562, "vocab_size": 50257, "entropy": 0.8206838965415955, "entropy_per_token": [0.5791852474212646, 0.6758645176887512, 0.09277608245611191, 1.976672649383545, 0.8828973770141602, 0.47115859389305115, 0.027132943272590637, 0.7868832349777222, 0.0004897878970950842, 1.279968023300171, 0.41755667328834534, 0.23114901781082153, 2.166538715362549, 1.0686674118041992, 0.5796384811401367, 0.7507957816123962, 1.328393816947937, 0.0023241627495735884, 3.0955610275268555, 2.4158740416169167e-05], "max_p": 0.7237038612365723, "max_p_per_token": [0.8503381609916687, 0.8192756175994873, 0.985268235206604, 0.37487658858299255, 0.6054461002349854, 0.8236470222473145, 0.9965597987174988, 0.7289692759513855, 0.9999626874923706, 0.4661576747894287, 0.9183312654495239, 0.9397402405738831, 0.24917955696582794, 0.7354441285133362, 0.8570532202720642, 0.5294218063354492, 0.48308461904525757, 0.9997740387916565, 0.11154845356941223, 0.9999984502792358], "n_positions_probed": 1, "per_restart_best": [7.9451165199279785]}
+{"step": 170, "discrete_loss": 8.235953330993652, "best_sample_loss": 7.9451165199279785, "soft_loss": 6.145585060119629, "best_discrete": 7.9451165199279785, "best_soft": 6.118430137634277, "best_argmax": 8.235953330993652, "best_sampling": 7.9451165199279785, "relax_gap": 0.2538101160684727, "n_match": 4, "g_first_norm": 157.4736785888672, "vocab_size": 50257, "entropy": 0.7960172295570374, "entropy_per_token": [0.5446687340736389, 0.6956548690795898, 0.09125164896249771, 1.936797857284546, 0.8906687498092651, 0.47559893131256104, 0.026786629110574722, 0.7951009273529053, 0.0004968420835211873, 1.1693272590637207, 0.015495412051677704, 0.24205628037452698, 2.196620464324951, 1.080511450767517, 0.5680422782897949, 0.7577123045921326, 1.3196229934692383, 0.002409183420240879, 3.1114959716796875, 2.5138751880149357e-05], "max_p": 0.7312635779380798, "max_p_per_token": [0.8630021810531616, 0.8118226528167725, 0.9855880737304688, 0.3951210081577301, 0.5919808149337769, 0.8207140564918518, 0.996616542339325, 0.7223048210144043, 0.9999622106552124, 0.5526230931282043, 0.9983413219451904, 0.9358204007148743, 0.23912832140922546, 0.7308151721954346, 0.861283540725708, 0.530441403388977, 0.4884004592895508, 0.9997655749320984, 0.10154106467962265, 0.9999983310699463], "n_positions_probed": 1, "per_restart_best": [7.9451165199279785]}
+{"step": 171, "discrete_loss": 8.235953330993652, "best_sample_loss": 8.037830352783203, "soft_loss": 6.0845441818237305, "best_discrete": 7.9451165199279785, "best_soft": 6.0845441818237305, "best_argmax": 8.235953330993652, "best_sampling": 7.9451165199279785, "relax_gap": 0.2612216294467951, "n_match": 4, "g_first_norm": 155.83091735839844, "vocab_size": 50257, "entropy": 0.8044729232788086, "entropy_per_token": [0.5385997295379639, 0.7094810605049133, 0.08841928839683533, 1.9146308898925781, 0.8856104612350464, 0.47675812244415283, 0.026846639811992645, 0.8033032417297363, 0.0004963473184034228, 1.2455793619155884, 0.015946989879012108, 0.2592119574546814, 2.235842227935791, 1.0903067588806152, 0.560116708278656, 0.7652587294578552, 1.339537501335144, 0.0024516484700143337, 3.1310338973999023, 2.6326444640289992e-05], "max_p": 0.7277005314826965, "max_p_per_token": [0.8660317659378052, 0.8071322441101074, 0.9861581325531006, 0.4042188227176666, 0.5996485948562622, 0.8199153542518616, 0.996613085269928, 0.7152565717697144, 0.9999622106552124, 0.5084275603294373, 0.9982878565788269, 0.9294796586036682, 0.23070694506168365, 0.727355420589447, 0.8639261722564697, 0.5300053954124451, 0.46765246987342834, 0.9997615218162537, 0.10347210615873337, 0.9999982118606567], "n_positions_probed": 1, "per_restart_best": [7.9451165199279785]}
+{"step": 172, "discrete_loss": 8.235953330993652, "best_sample_loss": 7.726868629455566, "soft_loss": 6.067636489868164, "best_discrete": 7.726868629455566, "best_soft": 6.067636489868164, "best_argmax": 8.235953330993652, "best_sampling": 7.726868629455566, "relax_gap": 0.2632745419969354, "n_match": 4, "g_first_norm": 158.6163787841797, "vocab_size": 50257, "entropy": 0.802463710308075, "entropy_per_token": [0.50918048620224, 0.7216759324073792, 0.08628113567829132, 1.8839499950408936, 0.8919734358787537, 0.4787070155143738, 0.026803627610206604, 0.8117663860321045, 0.0005017535877414048, 1.102009654045105, 0.016267001628875732, 0.27272576093673706, 2.3330986499786377, 1.1025868654251099, 0.5509665012359619, 0.7743021249771118, 1.3371895551681519, 0.0025767716579139233, 3.1466848850250244, 2.730804953898769e-05], "max_p": 0.7314240336418152, "max_p_per_token": [0.876244843006134, 0.8027114868164062, 0.9865874648094177, 0.4190249443054199, 0.5887956619262695, 0.8185847997665405, 0.9966249465942383, 0.7076650857925415, 0.9999617338180542, 0.6063515543937683, 0.9982512593269348, 0.924294650554657, 0.21990032494068146, 0.7225326895713806, 0.8671880960464478, 0.5314916968345642, 0.4641110301017761, 0.999748170375824, 0.09841158986091614, 0.9999982118606567], "n_positions_probed": 1, "per_restart_best": [7.726868629455566]}
+{"step": 173, "discrete_loss": 8.235953330993652, "best_sample_loss": 8.903973579406738, "soft_loss": 6.060210227966309, "best_discrete": 7.726868629455566, "best_soft": 6.060210227966309, "best_argmax": 8.235953330993652, "best_sampling": 7.726868629455566, "relax_gap": 0.2641762301929951, "n_match": 4, "g_first_norm": 153.18911743164062, "vocab_size": 50257, "entropy": 0.8101086616516113, "entropy_per_token": [0.5095618367195129, 0.7246849536895752, 0.08288850635290146, 1.8818035125732422, 0.8857961893081665, 0.4778369069099426, 0.02722216583788395, 0.8213576078414917, 0.0005028537125326693, 1.1299861669540405, 0.016861356794834137, 0.2942464351654053, 2.3887033462524414, 1.1094944477081299, 0.5445478558540344, 0.7824188470840454, 1.342217206954956, 0.002673634560778737, 3.179340362548828, 2.8528424081741832e-05], "max_p": 0.729978621006012, "max_p_per_token": [0.8768376111984253, 0.8024837374687195, 0.9872475862503052, 0.41869208216667175, 0.6009406447410583, 0.8191102743148804, 0.9965673685073853, 0.6995980143547058, 0.9999616146087646, 0.5947951674461365, 0.9981800317764282, 0.9156720638275146, 0.21104027330875397, 0.7200974822044373, 0.869201123714447, 0.5389696955680847, 0.454475075006485, 0.9997376799583435, 0.09596629440784454, 0.9999980926513672], "n_positions_probed": 1, "per_restart_best": [7.726868629455566]}
+{"step": 174, "discrete_loss": 8.235953330993652, "best_sample_loss": 8.660922050476074, "soft_loss": 6.026232719421387, "best_discrete": 7.726868629455566, "best_soft": 6.026232719421387, "best_argmax": 8.235953330993652, "best_sampling": 7.726868629455566, "relax_gap": 0.26830174028021925, "n_match": 4, "g_first_norm": 151.6444549560547, "vocab_size": 50257, "entropy": 0.8168014883995056, "entropy_per_token": [0.505821943283081, 0.7253627777099609, 0.07980488240718842, 1.8733410835266113, 0.8864514827728271, 0.47759467363357544, 0.027525920420885086, 0.8315558433532715, 0.0005050063482485712, 1.1534861326217651, 0.017390090972185135, 0.3148740530014038, 2.433655023574829, 1.1176586151123047, 0.5518670082092285, 0.7922744750976562, 1.3453551530838013, 0.002818810986354947, 3.1986570358276367, 2.958608092740178e-05], "max_p": 0.7276790738105774, "max_p_per_token": [0.8786299824714661, 0.8030623197555542, 0.9878405928611755, 0.42145615816116333, 0.6021578311920166, 0.8192273378372192, 0.9965260624885559, 0.6905771493911743, 0.9999614953994751, 0.5854814648628235, 0.9981170892715454, 0.9070211052894592, 0.19416770339012146, 0.7169191837310791, 0.8695858716964722, 0.5418561100959778, 0.44522401690483093, 0.9997214674949646, 0.0960504412651062, 0.9999979734420776], "n_positions_probed": 1, "per_restart_best": [7.726868629455566]}
+{"step": 175, "discrete_loss": 8.785064697265625, "best_sample_loss": 7.7288498878479, "soft_loss": 5.998547554016113, "best_discrete": 7.726868629455566, "best_soft": 5.998547554016113, "best_argmax": 8.235953330993652, "best_sampling": 7.726868629455566, "relax_gap": 0.3171880047868996, "n_match": 4, "g_first_norm": 151.60006713867188, "vocab_size": 50257, "entropy": 0.8237022757530212, "entropy_per_token": [0.508215606212616, 0.7229846119880676, 0.07683630287647247, 1.8644392490386963, 0.887836217880249, 0.47724223136901855, 0.027816304937005043, 0.8414259552955627, 0.0005064052529633045, 1.179229736328125, 0.017848659306764603, 0.3359876275062561, 2.4739108085632324, 1.12590754032135, 0.5457013845443726, 0.824916422367096, 1.3419314622879028, 0.002993534551933408, 3.218282699584961, 3.055761771975085e-05], "max_p": 0.7260977625846863, "max_p_per_token": [0.8783695101737976, 0.8048832416534424, 0.9884030222892761, 0.42426666617393494, 0.6023775339126587, 0.8194228410720825, 0.9964860677719116, 0.6815973520278931, 0.9999613761901855, 0.5746673345565796, 0.9980629086494446, 0.8977502584457397, 0.1957329511642456, 0.7136934399604797, 0.8715720176696777, 0.5407962799072266, 0.44037148356437683, 0.9997015595436096, 0.09384102374315262, 0.9999979734420776], "n_positions_probed": 1, "per_restart_best": [7.726868629455566]}
+{"step": 176, "discrete_loss": 8.785064697265625, "best_sample_loss": 8.004287719726562, "soft_loss": 5.970037460327148, "best_discrete": 7.726868629455566, "best_soft": 5.970037460327148, "best_argmax": 8.235953330993652, "best_sampling": 7.726868629455566, "relax_gap": 0.3204332960478551, "n_match": 4, "g_first_norm": 153.0968017578125, "vocab_size": 50257, "entropy": 0.8283122181892395, "entropy_per_token": [0.5112234950065613, 0.7224541902542114, 0.07405933737754822, 1.853393316268921, 0.8902971744537354, 0.47660112380981445, 0.028082724660634995, 0.8511748313903809, 0.0005071308114565909, 1.2047028541564941, 0.018223082646727562, 0.35727864503860474, 2.509810209274292, 1.1346060037612915, 0.5401827692985535, 0.8397248387336731, 1.3198223114013672, 0.0031976415775716305, 3.2308709621429443, 3.164984445902519e-05], "max_p": 0.7245882749557495, "max_p_per_token": [0.8779318332672119, 0.8058579564094543, 0.9889233112335205, 0.4279820919036865, 0.6008884310722351, 0.8198143243789673, 0.9964492321014404, 0.6721655130386353, 0.999961256980896, 0.563817024230957, 0.9980192184448242, 0.8879528641700745, 0.19954097270965576, 0.7102341651916504, 0.8733814358711243, 0.5391435027122498, 0.4359142780303955, 0.9996780157089233, 0.09411194175481796, 0.9999978542327881], "n_positions_probed": 1, "per_restart_best": [7.726868629455566]}
+{"step": 177, "discrete_loss": 8.785064697265625, "best_sample_loss": 7.67620849609375, "soft_loss": 5.940552234649658, "best_discrete": 7.67620849609375, "best_soft": 5.940552234649658, "best_argmax": 8.235953330993652, "best_sampling": 7.67620849609375, "relax_gap": 0.3237895861485606, "n_match": 4, "g_first_norm": 154.34674072265625, "vocab_size": 50257, "entropy": 0.8468387722969055, "entropy_per_token": [0.516924262046814, 0.7216947078704834, 0.07149015367031097, 1.8412946462631226, 0.8939290046691895, 0.47575849294662476, 0.028329171240329742, 0.8609443306922913, 0.0005069561884738505, 1.2242993116378784, 0.018492117524147034, 0.37858548760414124, 2.5403037071228027, 1.1438629627227783, 0.5347709655761719, 0.8568353652954102, 1.3150379657745361, 0.26970821619033813, 3.243974208831787, 3.2761650800239295e-05], "max_p": 0.7192158699035645, "max_p_per_token": [0.87663334608078, 0.8068551421165466, 0.9894000887870789, 0.4321545958518982, 0.5976526141166687, 0.8203460574150085, 0.9964152574539185, 0.6622186899185181, 0.9999613761901855, 0.556829571723938, 0.9979888200759888, 0.8776747584342957, 0.20318204164505005, 0.7065137624740601, 0.8751723170280457, 0.5368683338165283, 0.43153610825538635, 0.9237895607948303, 0.09312804043292999, 0.9999977350234985], "n_positions_probed": 1, "per_restart_best": [7.67620849609375]}
+{"step": 178, "discrete_loss": 9.057125091552734, "best_sample_loss": 7.435232639312744, "soft_loss": 5.887433052062988, "best_discrete": 7.435232639312744, "best_soft": 5.887433052062988, "best_argmax": 8.235953330993652, "best_sampling": 7.435232639312744, "relax_gap": 0.34996668451074037, "n_match": 4, "g_first_norm": 155.8343048095703, "vocab_size": 50257, "entropy": 0.8396829962730408, "entropy_per_token": [0.5221418142318726, 0.7251308560371399, 0.06910428404808044, 1.8245359659194946, 0.8974388837814331, 0.47428667545318604, 0.028594160452485085, 0.8698539733886719, 0.0005064284196123481, 1.2181488275527954, 0.018677441403269768, 0.4013690948486328, 2.5666778087615967, 1.1521852016448975, 0.5291841626167297, 0.8773948550224304, 1.3117547035217285, 0.30073514580726624, 3.005906105041504, 3.3870375773403794e-05], "max_p": 0.72391676902771, "max_p_per_token": [0.8755815625190735, 0.8060924410820007, 0.9898391962051392, 0.4383760392665863, 0.5943455696105957, 0.8212968707084656, 0.9963787198066711, 0.6524153351783752, 0.999961256980896, 0.566028892993927, 0.9979687333106995, 0.8661214709281921, 0.20530758798122406, 0.7030430436134338, 0.8770613074302673, 0.5292043089866638, 0.42351990938186646, 0.9109150171279907, 0.2248803675174713, 0.9999977350234985], "n_positions_probed": 1, "per_restart_best": [7.435232639312744]}
+{"step": 179, "discrete_loss": 9.723840713500977, "best_sample_loss": 8.401863098144531, "soft_loss": 6.671330451965332, "best_discrete": 7.435232639312744, "best_soft": 5.887433052062988, "best_argmax": 8.235953330993652, "best_sampling": 7.435232639312744, "relax_gap": 0.3139202246800912, "n_match": 4, "g_first_norm": 234.2778778076172, "vocab_size": 50257, "entropy": 0.853371798992157, "entropy_per_token": [0.5256584286689758, 0.7444602251052856, 0.06631596386432648, 1.8397564888000488, 0.9133119583129883, 0.46841731667518616, 0.028971388936042786, 0.8530817627906799, 0.0004864699440076947, 1.2408915758132935, 0.018320849165320396, 0.4254111051559448, 2.589388370513916, 1.151106357574463, 0.5291502475738525, 0.9051249027252197, 1.3746885061264038, 0.3235572576522827, 3.0692994594573975, 3.642176670837216e-05], "max_p": 0.7143154740333557, "max_p_per_token": [0.8754560947418213, 0.7989277839660645, 0.9903464913368225, 0.4318670928478241, 0.5735422968864441, 0.8251113891601562, 0.9963292479515076, 0.6640998721122742, 0.9999630451202393, 0.556823194026947, 0.9980157613754272, 0.8531162142753601, 0.20782260596752167, 0.7040586471557617, 0.8770517706871033, 0.498739629983902, 0.40840578079223633, 0.9008504748344421, 0.12578417360782623, 0.9999974966049194], "n_positions_probed": 1, "per_restart_best": [7.435232639312744]}
+{"step": 180, "discrete_loss": 9.6344633102417, "best_sample_loss": 7.493481636047363, "soft_loss": 6.083942413330078, "best_discrete": 7.435232639312744, "best_soft": 5.887433052062988, "best_argmax": 8.235953330993652, "best_sampling": 7.435232639312744, "relax_gap": 0.36852295582851197, "n_match": 4, "g_first_norm": 173.95974731445312, "vocab_size": 50257, "entropy": 0.8634477853775024, "entropy_per_token": [0.5606052279472351, 0.7111554145812988, 0.06315244734287262, 1.8282535076141357, 0.9199661612510681, 0.46371448040008545, 0.028632357716560364, 0.8682279586791992, 0.0004877327592112124, 1.2707322835922241, 0.018378354609012604, 0.4436081349849701, 2.5910158157348633, 1.1658625602722168, 0.528026282787323, 0.9289005994796753, 1.3602508306503296, 0.37625932693481445, 3.1416876316070557, 3.708211443154141e-05], "max_p": 0.7101894617080688, "max_p_per_token": [0.8333091139793396, 0.812667965888977, 0.9909002780914307, 0.4337925910949707, 0.5656476616859436, 0.8281105160713196, 0.9963807463645935, 0.6494635343551636, 0.9999630451202393, 0.5446394085884094, 0.9980114698410034, 0.842875599861145, 0.2172195017337799, 0.6979455947875977, 0.8774659633636475, 0.5154154896736145, 0.40882688760757446, 0.8754649758338928, 0.11569051444530487, 0.9999974966049194], "n_positions_probed": 1, "per_restart_best": [7.435232639312744]}
+{"step": 181, "discrete_loss": 9.69713306427002, "best_sample_loss": 7.466230869293213, "soft_loss": 5.926781177520752, "best_discrete": 7.435232639312744, "best_soft": 5.887433052062988, "best_argmax": 8.235953330993652, "best_sampling": 7.435232639312744, "relax_gap": 0.38881098792399543, "n_match": 4, "g_first_norm": 177.73324584960938, "vocab_size": 50257, "entropy": 0.8749408721923828, "entropy_per_token": [0.5412753224372864, 0.8747195601463318, 0.06055045500397682, 1.7996913194656372, 0.9252417087554932, 0.45715081691741943, 0.028850752860307693, 0.8807717561721802, 0.00048816949129104614, 1.1826417446136475, 0.01828675903379917, 0.47379374504089355, 2.6170382499694824, 1.1675496101379395, 0.5278225541114807, 0.9561023712158203, 1.3578675985336304, 0.41895079612731934, 3.2099852561950684, 3.844806633424014e-05], "max_p": 0.6975709199905396, "max_p_per_token": [0.8431475162506104, 0.5809294581413269, 0.9913581609725952, 0.44461488723754883, 0.5567514896392822, 0.8322293758392334, 0.9963504076004028, 0.6352815628051758, 0.9999629259109497, 0.6002854704856873, 0.9980252981185913, 0.8247421383857727, 0.20895002782344818, 0.696560800075531, 0.8776301145553589, 0.5072294473648071, 0.4108567535877228, 0.8524065613746643, 0.09410858899354935, 0.9999973773956299], "n_positions_probed": 1, "per_restart_best": [7.435232639312744]}
+{"step": 182, "discrete_loss": 9.6344633102417, "best_sample_loss": 7.437930583953857, "soft_loss": 5.919943809509277, "best_discrete": 7.435232639312744, "best_soft": 5.887433052062988, "best_argmax": 8.235953330993652, "best_sampling": 7.435232639312744, "relax_gap": 0.38554503568286835, "n_match": 4, "g_first_norm": 201.48257446289062, "vocab_size": 50257, "entropy": 0.8894514441490173, "entropy_per_token": [0.6184225082397461, 0.7968339920043945, 0.033838145434856415, 1.836201786994934, 0.9322508573532104, 0.4473356306552887, 0.029062218964099884, 0.8955626487731934, 0.0004947835695929825, 1.278362512588501, 0.018629901111125946, 0.49794164299964905, 2.654266357421875, 1.1758627891540527, 0.5287512540817261, 0.9896777868270874, 1.341484546661377, 0.4734812080860138, 3.2405290603637695, 3.9329555875156075e-05], "max_p": 0.6934401392936707, "max_p_per_token": [0.808592677116394, 0.6559041738510132, 0.9956603646278381, 0.42564019560813904, 0.5621271133422852, 0.8382628560066223, 0.9963162541389465, 0.6172382831573486, 0.9999624490737915, 0.5446130037307739, 0.9979846477508545, 0.8091562390327454, 0.19476990401744843, 0.6932591795921326, 0.8771060705184937, 0.5371350646018982, 0.3995385468006134, 0.8189452290534973, 0.09659403562545776, 0.9999972581863403], "n_positions_probed": 1, "per_restart_best": [7.435232639312744]}
+{"step": 183, "discrete_loss": 9.767581939697266, "best_sample_loss": 7.435232639312744, "soft_loss": 5.7803425788879395, "best_discrete": 7.435232639312744, "best_soft": 5.7803425788879395, "best_argmax": 8.235953330993652, "best_sampling": 7.435232639312744, "relax_gap": 0.4082115087874969, "n_match": 5, "g_first_norm": 182.0873260498047, "vocab_size": 50257, "entropy": 0.8025732040405273, "entropy_per_token": [0.584827184677124, 0.7793833613395691, 0.03285074234008789, 0.2035987377166748, 0.9340057373046875, 0.44271472096443176, 0.029359180480241776, 0.9028466939926147, 0.0004998208023607731, 1.1093719005584717, 0.018499203026294708, 0.541516900062561, 2.6514108180999756, 1.1788995265960693, 0.5254255533218384, 1.0324699878692627, 1.3334434032440186, 0.5200371146202087, 3.2302632331848145, 4.075727702002041e-05], "max_p": 0.7231736779212952, "max_p_per_token": [0.82569819688797, 0.667316734790802, 0.9958057403564453, 0.9662405848503113, 0.5583287477493286, 0.8410456776618958, 0.9962739944458008, 0.6047393083572388, 0.9999618530273438, 0.6430280208587646, 0.9980029463768005, 0.7779247760772705, 0.20541714131832123, 0.6912571787834167, 0.8785055875778198, 0.508598268032074, 0.422992467880249, 0.7857821583747864, 0.09655677527189255, 0.9999971389770508], "n_positions_probed": 1, "per_restart_best": [7.435232639312744]}
+{"step": 184, "discrete_loss": 8.72558879852295, "best_sample_loss": 7.741487979888916, "soft_loss": 5.725695610046387, "best_discrete": 7.435232639312744, "best_soft": 5.725695610046387, "best_argmax": 8.235953330993652, "best_sampling": 7.435232639312744, "relax_gap": 0.3438040982385485, "n_match": 5, "g_first_norm": 205.92526245117188, "vocab_size": 50257, "entropy": 0.8069246411323547, "entropy_per_token": [0.6346001625061035, 0.7366701364517212, 0.03238140791654587, 0.20719201862812042, 0.8818516731262207, 0.4299353361129761, 0.030689310282468796, 0.9204468727111816, 0.0005083256401121616, 1.1756024360656738, 0.01859210804104805, 0.5331805348396301, 2.615025281906128, 1.1848350763320923, 0.5200154781341553, 1.0497322082519531, 1.332131028175354, 0.5801640152931213, 3.2548985481262207, 4.1030143620446324e-05], "max_p": 0.7258588671684265, "max_p_per_token": [0.803632915019989, 0.7001878619194031, 0.9958738684654236, 0.9655143618583679, 0.6192595958709717, 0.848499596118927, 0.9960711002349854, 0.5778267979621887, 0.9999611377716064, 0.6069738268852234, 0.9979931116104126, 0.7848200798034668, 0.23244282603263855, 0.687964677810669, 0.8797857761383057, 0.5505298972129822, 0.4487442076206207, 0.7336961627006531, 0.08740183711051941, 0.9999971389770508], "n_positions_probed": 1, "per_restart_best": [7.435232639312744]}
+{"step": 185, "discrete_loss": 8.72558879852295, "best_sample_loss": 7.455260753631592, "soft_loss": 5.617926120758057, "best_discrete": 7.435232639312744, "best_soft": 5.617926120758057, "best_argmax": 8.235953330993652, "best_sampling": 7.435232639312744, "relax_gap": 0.35615506867467234, "n_match": 5, "g_first_norm": 171.44496154785156, "vocab_size": 50257, "entropy": 0.8145546913146973, "entropy_per_token": [0.6187218427658081, 0.7210109829902649, 0.03170259669423103, 0.21219474077224731, 0.873115062713623, 0.4184398055076599, 0.03141896426677704, 0.9202902913093567, 0.0005177279817871749, 1.224602460861206, 0.01861642301082611, 0.5766627192497253, 2.657332420349121, 1.2019150257110596, 0.513953685760498, 1.1004383563995361, 1.3460967540740967, 0.6041902899742126, 3.2198309898376465, 4.2778312490554526e-05], "max_p": 0.7192431688308716, "max_p_per_token": [0.8128125667572021, 0.706976592540741, 0.995974600315094, 0.9645123481750488, 0.6286953091621399, 0.8551490902900696, 0.9959564805030823, 0.5738093852996826, 0.9999603033065796, 0.5811653733253479, 0.9979920387268066, 0.7501996755599976, 0.21250587701797485, 0.6807861328125, 0.8820534348487854, 0.5034546256065369, 0.41337740421295166, 0.7084351778030396, 0.1210494413971901, 0.9999970197677612], "n_positions_probed": 1, "per_restart_best": [7.435232639312744]}
+{"step": 186, "discrete_loss": 8.72558879852295, "best_sample_loss": 7.435232639312744, "soft_loss": 5.508608818054199, "best_discrete": 7.435232639312744, "best_soft": 5.508608818054199, "best_argmax": 8.235953330993652, "best_sampling": 7.435232639312744, "relax_gap": 0.3686834269583405, "n_match": 5, "g_first_norm": 164.50164794921875, "vocab_size": 50257, "entropy": 0.8267480731010437, "entropy_per_token": [0.6404563784599304, 0.7034004330635071, 0.031155481934547424, 0.21520665287971497, 0.8580849170684814, 0.407694548368454, 0.03985007852315903, 0.9280179738998413, 0.0005253779818303883, 1.3559901714324951, 0.018603099510073662, 0.5943341255187988, 2.642141580581665, 1.2226283550262451, 0.5041064620018005, 1.1470686197280884, 1.3433797359466553, 0.6368111371994019, 3.2454631328582764, 4.39348368672654e-05], "max_p": 0.7121531367301941, "max_p_per_token": [0.8042290806770325, 0.7168713212013245, 0.9960536956787109, 0.9639507532119751, 0.647585391998291, 0.8610484600067139, 0.9949296712875366, 0.5592746734619141, 0.9999597072601318, 0.5060649514198303, 0.9979947805404663, 0.7352986931800842, 0.22544965147972107, 0.6718665957450867, 0.8853104710578918, 0.4912499785423279, 0.4096386432647705, 0.6672137379646301, 0.10907536000013351, 0.9999969005584717], "n_positions_probed": 1, "per_restart_best": [7.435232639312744]}
+{"step": 187, "discrete_loss": 8.740943908691406, "best_sample_loss": 7.471851348876953, "soft_loss": 5.467863082885742, "best_discrete": 7.435232639312744, "best_soft": 5.467863082885742, "best_argmax": 8.235953330993652, "best_sampling": 7.435232639312744, "relax_gap": 0.37445393312170017, "n_match": 5, "g_first_norm": 170.69505310058594, "vocab_size": 50257, "entropy": 0.8064813017845154, "entropy_per_token": [0.6323899030685425, 0.6969064474105835, 0.030763546004891396, 0.21822229027748108, 0.8449039459228516, 0.39935246109962463, 0.040926653891801834, 0.5414366126060486, 0.0005358572816476226, 1.263667345046997, 0.018557263538241386, 0.633682906627655, 2.6618475914001465, 1.236460566520691, 0.4937642216682434, 1.190990686416626, 1.3378212451934814, 0.6588373780250549, 3.228513717651367, 4.513973544817418e-05], "max_p": 0.726054847240448, "max_p_per_token": [0.8089210987091064, 0.7186954021453857, 0.9961101412773132, 0.9634202718734741, 0.6602726578712463, 0.8655121922492981, 0.9947651624679565, 0.8529922366142273, 0.999958872795105, 0.5686702132225037, 0.9980011582374573, 0.6966046094894409, 0.21598029136657715, 0.6655182242393494, 0.8886967897415161, 0.47470688819885254, 0.40992218255996704, 0.6315470337867737, 0.11080411076545715, 0.9999969005584717], "n_positions_probed": 1, "per_restart_best": [7.435232639312744]}
+{"step": 188, "discrete_loss": 8.740943908691406, "best_sample_loss": 7.384117126464844, "soft_loss": 5.479959964752197, "best_discrete": 7.384117126464844, "best_soft": 5.467863082885742, "best_argmax": 8.235953330993652, "best_sampling": 7.384117126464844, "relax_gap": 0.37306999999127166, "n_match": 5, "g_first_norm": 177.35227966308594, "vocab_size": 50257, "entropy": 0.8164209723472595, "entropy_per_token": [0.6390281915664673, 0.6941981315612793, 0.030469421297311783, 0.2182629108428955, 0.8171656727790833, 0.3831241726875305, 0.042595330625772476, 0.5335677862167358, 0.0703703761100769, 1.4321198463439941, 0.018391309306025505, 0.6283227801322937, 2.6839044094085693, 1.2573951482772827, 0.48272261023521423, 1.18912672996521, 1.339688777923584, 0.6312531232833862, 3.236666202545166, 4.6613087761215866e-05], "max_p": 0.724678635597229, "max_p_per_token": [0.8067861199378967, 0.7178429365158081, 0.9961536526679993, 0.963499128818512, 0.6855343580245972, 0.8739650249481201, 0.9945060610771179, 0.8556758761405945, 0.9907839298248291, 0.4676084816455841, 0.9980218410491943, 0.7042385935783386, 0.20584100484848022, 0.6559463143348694, 0.8920954465866089, 0.4894021451473236, 0.4027268886566162, 0.6750049591064453, 0.11794351786375046, 0.9999967813491821], "n_positions_probed": 1, "per_restart_best": [7.384117126464844]}
+{"step": 189, "discrete_loss": 8.740943908691406, "best_sample_loss": 7.420327663421631, "soft_loss": 5.471471786499023, "best_discrete": 7.384117126464844, "best_soft": 5.467863082885742, "best_argmax": 8.235953330993652, "best_sampling": 7.384117126464844, "relax_gap": 0.3740410825587657, "n_match": 5, "g_first_norm": 166.70648193359375, "vocab_size": 50257, "entropy": 0.8265169262886047, "entropy_per_token": [0.6180291175842285, 0.6985794305801392, 0.030357621610164642, 0.21825826168060303, 0.7952852845191956, 0.37598761916160583, 0.04371681064367294, 0.5250319242477417, 0.07043544948101044, 1.540226936340332, 0.018529381603002548, 0.661472737789154, 2.6947994232177734, 1.2764897346496582, 0.4732694923877716, 1.2268599271774292, 1.3400465250015259, 0.6760892868041992, 3.2468252182006836, 4.7639088734285906e-05], "max_p": 0.7188200950622559, "max_p_per_token": [0.8177676200866699, 0.7110753655433655, 0.9961697459220886, 0.9636178016662598, 0.7022419571876526, 0.8775798678398132, 0.9943301677703857, 0.8586218357086182, 0.9907684922218323, 0.46542686223983765, 0.9980060458183289, 0.6665422320365906, 0.2085685133934021, 0.6466779708862305, 0.8950220942497253, 0.4721333682537079, 0.4112953841686249, 0.5941099524497986, 0.10644955188035965, 0.9999966621398926], "n_positions_probed": 1, "per_restart_best": [7.384117126464844]}
+{"step": 190, "discrete_loss": 8.740943908691406, "best_sample_loss": 7.30450439453125, "soft_loss": 5.451269626617432, "best_discrete": 7.30450439453125, "best_soft": 5.451269626617432, "best_argmax": 8.235953330993652, "best_sampling": 7.30450439453125, "relax_gap": 0.3763522928917258, "n_match": 5, "g_first_norm": 195.00384521484375, "vocab_size": 50257, "entropy": 0.8214080929756165, "entropy_per_token": [0.6053865551948547, 0.6963323354721069, 0.030156001448631287, 0.2181941121816635, 0.7952178716659546, 0.3607105016708374, 0.045258983969688416, 0.5146939158439636, 0.07010407745838165, 1.5161869525909424, 0.07786371558904648, 0.6141365170478821, 2.7057456970214844, 1.2940274477005005, 0.46007248759269714, 1.2255042791366577, 1.3377394676208496, 0.6228236556053162, 3.23795747756958, 4.907119000563398e-05], "max_p": 0.7274157404899597, "max_p_per_token": [0.8228495717048645, 0.7107793688774109, 0.996198832988739, 0.9637734293937683, 0.7049696445465088, 0.8851062059402466, 0.9940869808197021, 0.8622620701789856, 0.9908055663108826, 0.4846634268760681, 0.9879742860794067, 0.7209610939025879, 0.20001430809497833, 0.6376988291740417, 0.8991581797599792, 0.47511330246925354, 0.4062226116657257, 0.6861664056777954, 0.11951399594545364, 0.999996542930603], "n_positions_probed": 1, "per_restart_best": [7.30450439453125]}
+{"step": 191, "discrete_loss": 8.878466606140137, "best_sample_loss": 7.30450439453125, "soft_loss": 5.4391188621521, "best_discrete": 7.30450439453125, "best_soft": 5.4391188621521, "best_argmax": 8.235953330993652, "best_sampling": 7.30450439453125, "relax_gap": 0.38738082785708355, "n_match": 6, "g_first_norm": 176.44503784179688, "vocab_size": 50257, "entropy": 0.7922137975692749, "entropy_per_token": [0.5825814604759216, 0.7003756761550903, 0.03002225235104561, 0.21733440458774567, 0.7993353009223938, 0.35362499952316284, 0.04672722890973091, 0.5073795914649963, 0.07037098705768585, 1.4106342792510986, 0.07746419310569763, 0.024356218054890633, 2.7409846782684326, 1.3104021549224854, 0.4508248567581177, 1.248863697052002, 1.3373936414718628, 0.6754599213600159, 3.26008939743042, 5.031671389588155e-05], "max_p": 0.7390109896659851, "max_p_per_token": [0.834194004535675, 0.7039322853088379, 0.9962185025215149, 0.9640632271766663, 0.7039221525192261, 0.888520359992981, 0.9938552975654602, 0.8647159934043884, 0.9907562136650085, 0.5485925674438477, 0.9880935549736023, 0.9963405132293701, 0.19376985728740692, 0.629115879535675, 0.9019486904144287, 0.46388179063796997, 0.41613489389419556, 0.5958121418952942, 0.10635513067245483, 0.9999964237213135], "n_positions_probed": 1, "per_restart_best": [7.30450439453125]}
+{"step": 192, "discrete_loss": 8.517143249511719, "best_sample_loss": 7.585788249969482, "soft_loss": 5.726752281188965, "best_discrete": 7.30450439453125, "best_soft": 5.4391188621521, "best_argmax": 8.235953330993652, "best_sampling": 7.30450439453125, "relax_gap": 0.32762052798427743, "n_match": 6, "g_first_norm": 245.65032958984375, "vocab_size": 50257, "entropy": 0.6573153138160706, "entropy_per_token": [0.5830146074295044, 0.6686649322509766, 0.030316051095724106, 0.21159398555755615, 0.8123418092727661, 0.3382980227470398, 0.048492684960365295, 0.5024358034133911, 0.07029733806848526, 1.671531081199646, 0.08059802651405334, 0.02736678160727024, 0.04687810316681862, 1.33143949508667, 0.43502476811408997, 1.1543405055999756, 1.3488311767578125, 0.5766626596450806, 3.2081246376037598, 5.273250280879438e-05], "max_p": 0.7839179039001465, "max_p_per_token": [0.8332952857017517, 0.7264747023582458, 0.9961762428283691, 0.9653031826019287, 0.6962010264396667, 0.8956683874130249, 0.9935611486434937, 0.866355836391449, 0.9907474517822266, 0.3840678036212921, 0.9875520467758179, 0.9958024621009827, 0.994573175907135, 0.6192162036895752, 0.9066706895828247, 0.5207481384277344, 0.4133910834789276, 0.7371048927307129, 0.1554512083530426, 0.9999963045120239], "n_positions_probed": 1, "per_restart_best": [7.30450439453125]}
+{"step": 193, "discrete_loss": 9.939518928527832, "best_sample_loss": 8.647943496704102, "soft_loss": 5.829160690307617, "best_discrete": 7.30450439453125, "best_soft": 5.4391188621521, "best_argmax": 8.235953330993652, "best_sampling": 7.30450439453125, "relax_gap": 0.4135369395417018, "n_match": 5, "g_first_norm": 140.6340789794922, "vocab_size": 50257, "entropy": 0.6417422294616699, "entropy_per_token": [0.5826035737991333, 0.6583386063575745, 0.031212475150823593, 0.2068835198879242, 0.8193846940994263, 0.33760297298431396, 0.04956439137458801, 0.4963815212249756, 0.07115921378135681, 1.7422587871551514, 0.08395988494157791, 0.030524620786309242, 0.05017929524183273, 0.7128099799156189, 0.4265213906764984, 1.2328414916992188, 1.3569974899291992, 0.6669531464576721, 3.2786128520965576, 5.3883799409959465e-05], "max_p": 0.7810609936714172, "max_p_per_token": [0.8345067501068115, 0.7316614389419556, 0.9960456490516663, 0.966304361820221, 0.693031370639801, 0.8959610462188721, 0.9933868050575256, 0.8682607412338257, 0.9906057715415955, 0.3500078618526459, 0.9869623780250549, 0.9952227473258972, 0.9941452741622925, 0.8034955859184265, 0.9090448021888733, 0.4609304666519165, 0.42347410321235657, 0.6154441833496094, 0.1127319261431694, 0.9999961853027344], "n_positions_probed": 1, "per_restart_best": [7.30450439453125]}
+{"step": 194, "discrete_loss": 9.939518928527832, "best_sample_loss": 7.30450439453125, "soft_loss": 6.5894880294799805, "best_discrete": 7.30450439453125, "best_soft": 5.4391188621521, "best_argmax": 8.235953330993652, "best_sampling": 7.30450439453125, "relax_gap": 0.33704155333240393, "n_match": 5, "g_first_norm": 251.8419189453125, "vocab_size": 50257, "entropy": 0.6351756453514099, "entropy_per_token": [0.579534113407135, 0.640843391418457, 0.0326823964715004, 0.20440417528152466, 0.8255616426467896, 0.33339837193489075, 0.05179772526025772, 0.48167717456817627, 0.07273957133293152, 1.8687248229980469, 0.08704646676778793, 0.03495500236749649, 0.06480896472930908, 0.9735516309738159, 0.0045651625841856, 1.2159395217895508, 1.3260046243667603, 0.6462199687957764, 3.258984088897705, 7.333698158618063e-05], "max_p": 0.777760922908783, "max_p_per_token": [0.8341574668884277, 0.7440553307533264, 0.9958350658416748, 0.9668257832527161, 0.6887650489807129, 0.8979277610778809, 0.9929860234260559, 0.873197615146637, 0.9903360605239868, 0.29176318645477295, 0.9864107966423035, 0.9943830966949463, 0.9921998381614685, 0.6117382645606995, 0.9994731545448303, 0.47422298789024353, 0.4719807207584381, 0.6530056595802307, 0.09596053510904312, 0.9999946355819702], "n_positions_probed": 1, "per_restart_best": [7.30450439453125]}
+{"step": 195, "discrete_loss": 8.58020305633545, "best_sample_loss": 7.180311679840088, "soft_loss": 6.006374835968018, "best_discrete": 7.180311679840088, "best_soft": 5.4391188621521, "best_argmax": 8.235953330993652, "best_sampling": 7.180311679840088, "relax_gap": 0.2999728798337667, "n_match": 6, "g_first_norm": 196.15919494628906, "vocab_size": 50257, "entropy": 0.6436344981193542, "entropy_per_token": [0.5724446773529053, 0.6265112161636353, 0.03425491973757744, 0.19475141167640686, 0.8544198274612427, 0.3255171775817871, 0.05214748904109001, 0.4723764657974243, 0.07454732060432434, 1.981642246246338, 0.08730218559503555, 0.041745107620954514, 0.08093886077404022, 0.9921419620513916, 0.005162442103028297, 1.2276321649551392, 1.3434193134307861, 0.665942370891571, 3.239699602127075, 9.367549500893801e-05], "max_p": 0.7730801701545715, "max_p_per_token": [0.8367177844047546, 0.7530236840248108, 0.9956045150756836, 0.9687156677246094, 0.6725850105285645, 0.9014973044395447, 0.9929087162017822, 0.8761587142944336, 0.990044116973877, 0.2710067629814148, 0.9863986372947693, 0.9930495023727417, 0.9899699091911316, 0.5899313688278198, 0.9993935823440552, 0.43174096941947937, 0.46362969279289246, 0.6174992918968201, 0.1317344754934311, 0.9999929666519165], "n_positions_probed": 1, "per_restart_best": [7.180311679840088]}
+{"step": 196, "discrete_loss": 8.58020305633545, "best_sample_loss": 7.115464210510254, "soft_loss": 5.525951385498047, "best_discrete": 7.115464210510254, "best_soft": 5.4391188621521, "best_argmax": 8.235953330993652, "best_sampling": 7.115464210510254, "relax_gap": 0.3559649638573768, "n_match": 7, "g_first_norm": 172.06842041015625, "vocab_size": 50257, "entropy": 0.5793851613998413, "entropy_per_token": [0.5735315084457397, 0.6197545528411865, 0.03519716113805771, 0.1890355348587036, 0.834985613822937, 0.31778484582901, 0.05272875726222992, 0.4683424234390259, 0.07579618692398071, 2.0743772983551025, 0.09016122668981552, 0.04760562255978584, 0.09082286059856415, 0.9803763628005981, 0.005038365256041288, 1.2378759384155273, 0.0019213458290323615, 0.6535782814025879, 3.2386903762817383, 9.822876018006355e-05], "max_p": 0.8018695116043091, "max_p_per_token": [0.8359741568565369, 0.7561076283454895, 0.9954645037651062, 0.9698765873908997, 0.6887311935424805, 0.9049140810966492, 0.9928056597709656, 0.8771311044692993, 0.9898362755775452, 0.26227256655693054, 0.9859090447425842, 0.9918577075004578, 0.9885644316673279, 0.6127249002456665, 0.9994101524353027, 0.40924742817878723, 0.9998231530189514, 0.6408350467681885, 0.13591095805168152, 0.9999926090240479], "n_positions_probed": 1, "per_restart_best": [7.115464210510254]}
+{"step": 197, "discrete_loss": 8.58020305633545, "best_sample_loss": 6.8357439041137695, "soft_loss": 5.595861434936523, "best_discrete": 6.8357439041137695, "best_soft": 5.4391188621521, "best_argmax": 8.235953330993652, "best_sampling": 6.8357439041137695, "relax_gap": 0.3478171322758321, "n_match": 7, "g_first_norm": 207.65411376953125, "vocab_size": 50257, "entropy": 0.57928466796875, "entropy_per_token": [0.543569803237915, 0.6059978008270264, 0.03629351034760475, 0.1856740415096283, 0.8321632146835327, 0.31715017557144165, 0.0541488341987133, 0.4585134983062744, 0.07881797850131989, 2.1576719284057617, 0.09327211230993271, 0.055246610194444656, 0.0999101847410202, 0.9543423056602478, 0.005030130967497826, 1.1987348794937134, 0.0022355541586875916, 0.693583607673645, 3.213233470916748, 0.00010345736518502235], "max_p": 0.7969341278076172, "max_p_per_token": [0.8471305966377258, 0.7641270160675049, 0.9953047037124634, 0.9705096483230591, 0.691161572933197, 0.9051809906959534, 0.9925655126571655, 0.8802406191825867, 0.9893452525138855, 0.23651504516601562, 0.9853715300559998, 0.9902447462081909, 0.9872559309005737, 0.6480547189712524, 0.99941086769104, 0.4180592894554138, 0.9997908473014832, 0.5128673911094666, 0.12555459141731262, 0.9999921321868896], "n_positions_probed": 1, "per_restart_best": [6.8357439041137695]}
+{"step": 198, "discrete_loss": 7.929473876953125, "best_sample_loss": 6.824120998382568, "soft_loss": 5.611634254455566, "best_discrete": 6.824120998382568, "best_soft": 5.4391188621521, "best_argmax": 7.929473876953125, "best_sampling": 6.824120998382568, "relax_gap": 0.29230686152259333, "n_match": 8, "g_first_norm": 289.0752258300781, "vocab_size": 50257, "entropy": 0.4945862889289856, "entropy_per_token": [0.5521938800811768, 0.5858690142631531, 0.03654344007372856, 0.17704923450946808, 0.8612313866615295, 0.3040504455566406, 0.050711214542388916, 0.4559439420700073, 0.08041390776634216, 2.2349469661712646, 0.09258334338665009, 0.0637066662311554, 0.11852733790874481, 0.9320625066757202, 0.004870980978012085, 1.1944901943206787, 0.002555454382672906, 0.2525360584259033, 1.8913297653198242, 0.0001095947518479079], "max_p": 0.8423177599906921, "max_p_per_token": [0.8423252105712891, 0.7764310240745544, 0.9952682852745056, 0.972187340259552, 0.6744359731674194, 0.910808265209198, 0.9930960536003113, 0.880704402923584, 0.9890828132629395, 0.2174544632434845, 0.985519289970398, 0.9883928894996643, 0.9845008850097656, 0.671402096748352, 0.9994319081306458, 0.43066591024398804, 0.9997571110725403, 0.9305707812309265, 0.6043302416801453, 0.9999916553497314], "n_positions_probed": 1, "per_restart_best": [6.824120998382568]}
+{"step": 199, "discrete_loss": 7.929473876953125, "best_sample_loss": 7.832263469696045, "soft_loss": 7.373822212219238, "best_discrete": 6.824120998382568, "best_soft": 5.4391188621521, "best_argmax": 7.929473876953125, "best_sampling": 6.824120998382568, "relax_gap": 0.07007421593869909, "n_match": 8, "g_first_norm": 196.64869689941406, "vocab_size": 50257, "entropy": 0.5284510850906372, "entropy_per_token": [0.6245085000991821, 0.558182954788208, 0.03732772916555405, 0.1772918850183487, 0.8572447299957275, 0.30618342757225037, 0.05019931122660637, 0.45471689105033875, 0.07796528190374374, 2.3101718425750732, 0.09353075176477432, 0.0693977102637291, 0.12852230668067932, 0.8721655607223511, 0.0050119333900511265, 1.1009962558746338, 0.0024985966738313437, 0.3196835219860077, 2.523298740386963, 0.00012319086818024516], "max_p": 0.8369215130805969, "max_p_per_token": [0.8120215535163879, 0.7935693860054016, 0.9951555728912354, 0.9721317887306213, 0.678062915802002, 0.909895122051239, 0.9931764006614685, 0.8805844783782959, 0.9894580841064453, 0.18587522208690643, 0.9853555560112, 0.987105131149292, 0.9829344153404236, 0.7155249714851379, 0.9994134902954102, 0.573630690574646, 0.9997630715370178, 0.9026483297348022, 0.3821331262588501, 0.9999904632568359], "n_positions_probed": 1, "per_restart_best": [6.824120998382568]}
+{"step": 200, "discrete_loss": 8.055414199829102, "best_sample_loss": 6.782452583312988, "soft_loss": 6.955824851989746, "best_discrete": 6.782452583312988, "best_soft": 5.4391188621521, "best_argmax": 7.929473876953125, "best_sampling": 6.782452583312988, "relax_gap": 0.13650314193188026, "n_match": 7, "g_first_norm": 204.12203979492188, "vocab_size": 50257, "entropy": 0.5543714761734009, "entropy_per_token": [0.7159823179244995, 0.5329307317733765, 0.03996725380420685, 0.1798839122056961, 0.8351920247077942, 0.31361642479896545, 0.05130591243505478, 0.46062320470809937, 0.07510251551866531, 2.4121575355529785, 0.10105125606060028, 0.07797996699810028, 0.1478719413280487, 0.8785190582275391, 0.005217221099883318, 1.0413874387741089, 0.002365408232435584, 0.37942931056022644, 2.8367180824279785, 0.0001273709931410849], "max_p": 0.8254770636558533, "max_p_per_token": [0.7674375176429749, 0.8092649579048157, 0.994763970375061, 0.9716438055038452, 0.6934004426002502, 0.9067296981811523, 0.993010938167572, 0.8776992559432983, 0.9898722171783447, 0.17524471879005432, 0.9839527010917664, 0.9851069450378418, 0.9798691868782043, 0.715929388999939, 0.999386191368103, 0.6290858387947083, 0.9997768998146057, 0.8739378452301025, 0.16343629360198975, 0.9999901056289673], "n_positions_probed": 1, "per_restart_best": [6.782452583312988]}
+{"step": 201, "discrete_loss": 7.937896728515625, "best_sample_loss": 6.705699443817139, "soft_loss": 5.8128204345703125, "best_discrete": 6.705699443817139, "best_soft": 5.4391188621521, "best_argmax": 7.929473876953125, "best_sampling": 6.705699443817139, "relax_gap": 0.26771276657093757, "n_match": 6, "g_first_norm": 198.5700225830078, "vocab_size": 50257, "entropy": 0.5635978579521179, "entropy_per_token": [0.6723757386207581, 0.5351870656013489, 0.04077179729938507, 0.1788426786661148, 0.8186780214309692, 0.3196842670440674, 0.05109141021966934, 0.4522935152053833, 0.07311201095581055, 2.4746413230895996, 0.10523171722888947, 0.08899238705635071, 0.15761631727218628, 0.8733019232749939, 0.005180490668863058, 1.0010147094726562, 0.0027041472494602203, 0.49477797746658325, 2.9263296127319336, 0.00012886534386780113], "max_p": 0.823864758014679, "max_p_per_token": [0.7906933426856995, 0.8123830556869507, 0.9946409463882446, 0.9718500375747681, 0.7048489451408386, 0.9040921330451965, 0.9930307269096375, 0.8802024126052856, 0.9901829361915588, 0.17098776996135712, 0.9831703305244446, 0.982439398765564, 0.9783331751823425, 0.7239962220191956, 0.9993913173675537, 0.6613729000091553, 0.9997410178184509, 0.8046272397041321, 0.13131967186927795, 0.9999899864196777], "n_positions_probed": 1, "per_restart_best": [6.705699443817139]}
+{"step": 202, "discrete_loss": 7.937896728515625, "best_sample_loss": 6.68490743637085, "soft_loss": 5.370566368103027, "best_discrete": 6.68490743637085, "best_soft": 5.370566368103027, "best_argmax": 7.929473876953125, "best_sampling": 6.68490743637085, "relax_gap": 0.32342702962988595, "n_match": 6, "g_first_norm": 164.40634155273438, "vocab_size": 50257, "entropy": 0.5762429237365723, "entropy_per_token": [0.6504783630371094, 0.53487229347229, 0.04469360411167145, 0.17846132814884186, 0.8047894239425659, 0.3246772289276123, 0.05162560194730759, 0.44751614332199097, 0.07370352745056152, 2.5167593955993652, 0.11075976490974426, 0.10286115109920502, 0.1698664426803589, 0.8570806384086609, 0.005307010840624571, 1.0186285972595215, 0.00314869056455791, 0.6449050307273865, 2.9845948219299316, 0.00012855646491516382], "max_p": 0.8167427182197571, "max_p_per_token": [0.8004546165466309, 0.8116888403892517, 0.9940521121025085, 0.971959114074707, 0.714954674243927, 0.901906430721283, 0.9929386973381042, 0.8814862370491028, 0.9900807738304138, 0.17746180295944214, 0.9821205735206604, 0.9789304733276367, 0.9763565063476562, 0.735706627368927, 0.9993744492530823, 0.6556816101074219, 0.9996930360794067, 0.6564714908599854, 0.11354553699493408, 0.9999899864196777], "n_positions_probed": 1, "per_restart_best": [6.68490743637085]}
+{"step": 203, "discrete_loss": 7.937896728515625, "best_sample_loss": 6.755160808563232, "soft_loss": 5.056194305419922, "best_discrete": 6.68490743637085, "best_soft": 5.056194305419922, "best_argmax": 7.929473876953125, "best_sampling": 6.68490743637085, "relax_gap": 0.36303097932020806, "n_match": 6, "g_first_norm": 150.30621337890625, "vocab_size": 50257, "entropy": 0.5854504704475403, "entropy_per_token": [0.6529178023338318, 0.53282630443573, 0.04581570252776146, 0.18761208653450012, 0.836600661277771, 0.324296236038208, 0.05226830393075943, 0.4433259665966034, 0.0756705105304718, 2.5698659420013428, 0.11363609880208969, 0.11839660257101059, 0.18700458109378815, 0.8392687439918518, 0.005339703056961298, 1.0109161138534546, 0.003595927031710744, 0.6961427927017212, 3.013381004333496, 0.00012894363317172974], "max_p": 0.808968186378479, "max_p_per_token": [0.7988634705543518, 0.8121953010559082, 0.9938806295394897, 0.9708105325698853, 0.6961652636528015, 0.9021019339561462, 0.9928299784660339, 0.8825643062591553, 0.989769458770752, 0.17360541224479675, 0.9815664291381836, 0.9748072028160095, 0.973545253276825, 0.7473890781402588, 0.9993700385093689, 0.6635196805000305, 0.9996436834335327, 0.5023588538169861, 0.1243884265422821, 0.9999899864196777], "n_positions_probed": 1, "per_restart_best": [6.68490743637085]}
+{"step": 204, "discrete_loss": 7.70297384262085, "best_sample_loss": 6.68490743637085, "soft_loss": 5.211212158203125, "best_discrete": 6.68490743637085, "best_soft": 5.056194305419922, "best_argmax": 7.70297384262085, "best_sampling": 6.68490743637085, "relax_gap": 0.3234804810877991, "n_match": 7, "g_first_norm": 263.4173278808594, "vocab_size": 50257, "entropy": 0.5238670706748962, "entropy_per_token": [0.6474467515945435, 0.5278439521789551, 0.045961279422044754, 0.18244636058807373, 0.007414871361106634, 0.3170345723628998, 0.04903049394488335, 0.44709593057632446, 0.07837359607219696, 2.6251890659332275, 0.11251343786716461, 0.13441289961338043, 0.21688178181648254, 0.8212925791740417, 0.005226559937000275, 1.0174728631973267, 0.004091276321560144, 0.23500211536884308, 3.0024776458740234, 0.00013216501974966377], "max_p": 0.8459742665290833, "max_p_per_token": [0.7999756336212158, 0.8146044611930847, 0.993860125541687, 0.9717901349067688, 0.9991457462310791, 0.9053037762641907, 0.9933326840400696, 0.880530059337616, 0.9893441200256348, 0.16395257413387299, 0.9817724227905273, 0.9703595638275146, 0.968532145023346, 0.7580590844154358, 0.9993851184844971, 0.6593071818351746, 0.9995877146720886, 0.9379420876502991, 0.1327110379934311, 0.9999897480010986], "n_positions_probed": 1, "per_restart_best": [6.68490743637085]}
+{"step": 205, "discrete_loss": 7.70297384262085, "best_sample_loss": 6.733328342437744, "soft_loss": 5.635725975036621, "best_discrete": 6.68490743637085, "best_soft": 5.056194305419922, "best_argmax": 7.70297384262085, "best_sampling": 6.68490743637085, "relax_gap": 0.26837010092726354, "n_match": 7, "g_first_norm": 233.02197265625, "vocab_size": 50257, "entropy": 0.5250740647315979, "entropy_per_token": [0.6029781699180603, 0.5234471559524536, 0.04581132531166077, 0.18195614218711853, 0.0076028648763895035, 0.309994101524353, 0.049393296241760254, 0.44397851824760437, 0.08035141974687576, 2.674043655395508, 0.11634371429681778, 0.15411357581615448, 0.2371201366186142, 0.792770266532898, 0.005075996275991201, 0.9631353616714478, 0.004969517234712839, 0.2909753918647766, 3.017292022705078, 0.00012811156921088696], "max_p": 0.8477838635444641, "max_p_per_token": [0.8218695521354675, 0.8172288537025452, 0.9938828945159912, 0.9718467593193054, 0.9991256594657898, 0.9094131588935852, 0.993279755115509, 0.881144642829895, 0.9890069961547852, 0.16243888437747955, 0.9810463190078735, 0.9645901918411255, 0.9651263356208801, 0.7716371417045593, 0.9994053840637207, 0.6932938098907471, 0.9994868040084839, 0.916638970375061, 0.12522496283054352, 0.9999899864196777], "n_positions_probed": 1, "per_restart_best": [6.68490743637085]}
+{"step": 206, "discrete_loss": 7.70297384262085, "best_sample_loss": 6.68490743637085, "soft_loss": 5.538619041442871, "best_discrete": 6.68490743637085, "best_soft": 5.056194305419922, "best_argmax": 7.70297384262085, "best_sampling": 6.68490743637085, "relax_gap": 0.2809765222364537, "n_match": 7, "g_first_norm": 216.05377197265625, "vocab_size": 50257, "entropy": 0.5708608031272888, "entropy_per_token": [0.5800817608833313, 0.5204941630363464, 0.04587339237332344, 0.18043102324008942, 0.00797906331717968, 0.29963764548301697, 0.8764923214912415, 0.44258949160575867, 0.08288970589637756, 2.720252513885498, 0.11997416615486145, 0.17494885623455048, 0.2598930299282074, 0.767806351184845, 0.00492622796446085, 0.9350333213806152, 0.006032698787748814, 0.36092260479927063, 3.0308337211608887, 0.00012429626076482236], "max_p": 0.8298603296279907, "max_p_per_token": [0.832575261592865, 0.8190125226974487, 0.993873119354248, 0.9721096158027649, 0.9990803003311157, 0.913826584815979, 0.6326503753662109, 0.8810972571372986, 0.9885744452476501, 0.16006894409656525, 0.9803544878959656, 0.9581508636474609, 0.9611889123916626, 0.7828814387321472, 0.9994252920150757, 0.7103686928749084, 0.999360978603363, 0.8865234851837158, 0.1260940134525299, 0.9999903440475464], "n_positions_probed": 1, "per_restart_best": [6.68490743637085]}
+{"step": 207, "discrete_loss": 7.70297384262085, "best_sample_loss": 6.721920013427734, "soft_loss": 5.480269432067871, "best_discrete": 6.68490743637085, "best_soft": 5.056194305419922, "best_argmax": 7.70297384262085, "best_sampling": 6.68490743637085, "relax_gap": 0.28855146803883325, "n_match": 7, "g_first_norm": 212.7654266357422, "vocab_size": 50257, "entropy": 0.5648379325866699, "entropy_per_token": [0.5700627565383911, 0.5186150074005127, 0.04600197449326515, 0.17848077416419983, 0.008579489775002003, 0.2874360978603363, 0.8731740117073059, 0.21677955985069275, 0.08610756695270538, 2.753343105316162, 0.12379494309425354, 0.2001143842935562, 0.2835429906845093, 0.7393962144851685, 0.004757497925311327, 0.9030612707138062, 0.007384184747934341, 0.4572262763977051, 3.038778781890869, 0.00012164862710051239], "max_p": 0.8321768045425415, "max_p_per_token": [0.8375339508056641, 0.8202223777770996, 0.9938515424728394, 0.9724708199501038, 0.9990049004554749, 0.918883204460144, 0.6339403986930847, 0.9476709365844727, 0.988029420375824, 0.15428362786769867, 0.9796055555343628, 0.9499103426933289, 0.9570048451423645, 0.7951354384422302, 0.999447762966156, 0.7279560565948486, 0.9991968274116516, 0.8375866413116455, 0.13181063532829285, 0.9999905824661255], "n_positions_probed": 1, "per_restart_best": [6.68490743637085]}
+{"step": 208, "discrete_loss": 7.70297384262085, "best_sample_loss": 6.6030707359313965, "soft_loss": 5.397913932800293, "best_discrete": 6.6030707359313965, "best_soft": 5.056194305419922, "best_argmax": 7.70297384262085, "best_sampling": 6.6030707359313965, "relax_gap": 0.29924285826683866, "n_match": 7, "g_first_norm": 218.1229705810547, "vocab_size": 50257, "entropy": 0.5728943943977356, "entropy_per_token": [0.5618138313293457, 0.5168636441230774, 0.04617456719279289, 0.1763356477022171, 0.009336707182228565, 0.27525022625923157, 0.872044026851654, 0.21416838467121124, 0.09516186267137527, 2.7820205688476562, 0.12771901488304138, 0.22622820734977722, 0.30949774384498596, 0.7109154462814331, 0.004565625451505184, 0.8789880275726318, 0.009112362749874592, 0.5854155421257019, 3.056157350540161, 0.00011954200454056263], "max_p": 0.8285879492759705, "max_p_per_token": [0.8416179418563843, 0.8213686347007751, 0.9938228130340576, 0.9728666543960571, 0.9989078044891357, 0.923801600933075, 0.6331897974014282, 0.9484263062477112, 0.9868005514144897, 0.14818185567855835, 0.9788287281990051, 0.940814733505249, 0.9523130059242249, 0.8070169687271118, 0.9994731545448303, 0.740533173084259, 0.9989801049232483, 0.7536304593086243, 0.13119208812713623, 0.999990701675415], "n_positions_probed": 1, "per_restart_best": [6.6030707359313965]}
+{"step": 209, "discrete_loss": 7.70297384262085, "best_sample_loss": 6.578393459320068, "soft_loss": 5.265749931335449, "best_discrete": 6.578393459320068, "best_soft": 5.056194305419922, "best_argmax": 7.70297384262085, "best_sampling": 6.578393459320068, "relax_gap": 0.3164003878346499, "n_match": 7, "g_first_norm": 235.90585327148438, "vocab_size": 50257, "entropy": 0.5851117968559265, "entropy_per_token": [0.555754542350769, 0.5148550271987915, 0.04614463075995445, 0.1740587204694748, 0.010541552677750587, 0.262307345867157, 0.8697883486747742, 0.21229475736618042, 0.09984423220157623, 2.905503273010254, 0.1315779685974121, 0.2531838119029999, 0.3397827744483948, 0.6784340143203735, 0.004295174963772297, 0.8437256813049316, 0.011419273912906647, 0.7205098867416382, 3.068096160888672, 0.00011763051588786766], "max_p": 0.8217137455940247, "max_p_per_token": [0.8448829650878906, 0.8227368593215942, 0.9938247203826904, 0.9732763767242432, 0.9987491369247437, 0.9288780689239502, 0.6333119869232178, 0.9489374756813049, 0.9859876036643982, 0.1231381744146347, 0.9780557155609131, 0.9308308959007263, 0.9467169642448425, 0.8200350999832153, 0.9995086193084717, 0.7571264505386353, 0.9986816048622131, 0.6158372163772583, 0.13376696407794952, 0.9999909400939941], "n_positions_probed": 1, "per_restart_best": [6.578393459320068]}
+{"step": 210, "discrete_loss": 7.817512035369873, "best_sample_loss": 6.578393459320068, "soft_loss": 5.084424018859863, "best_discrete": 6.578393459320068, "best_soft": 5.056194305419922, "best_argmax": 7.70297384262085, "best_sampling": 6.578393459320068, "relax_gap": 0.34961097650305034, "n_match": 8, "g_first_norm": 236.7804718017578, "vocab_size": 50257, "entropy": 0.6157185435295105, "entropy_per_token": [0.5573253035545349, 0.5107239484786987, 0.04573284089565277, 0.1738746464252472, 0.01139684859663248, 0.2473878413438797, 0.854356586933136, 0.2137354165315628, 0.10668906569480896, 2.9382565021514893, 0.7063618302345276, 0.2797192335128784, 0.37878888845443726, 0.6454359292984009, 0.0039380998350679874, 0.8027946352958679, 0.014046424068510532, 0.7628490924835205, 3.060842514038086, 0.00011616781557677314], "max_p": 0.8022859692573547, "max_p_per_token": [0.8447414040565491, 0.8253645300865173, 0.9938849806785583, 0.9732795357704163, 0.9986339211463928, 0.934535562992096, 0.646264910697937, 0.9483851790428162, 0.9848151206970215, 0.11421659588813782, 0.5843240022659302, 0.9203888177871704, 0.9393342733383179, 0.8326738476753235, 0.9995548129081726, 0.7750405669212341, 0.998330295085907, 0.5918152928352356, 0.14014415442943573, 0.9999910593032837], "n_positions_probed": 1, "per_restart_best": [6.578393459320068]}
+{"step": 211, "discrete_loss": 7.756961345672607, "best_sample_loss": 6.676425457000732, "soft_loss": 5.031754970550537, "best_discrete": 6.578393459320068, "best_soft": 5.031754970550537, "best_argmax": 7.70297384262085, "best_sampling": 6.578393459320068, "relax_gap": 0.35132395968975494, "n_match": 7, "g_first_norm": 267.5373840332031, "vocab_size": 50257, "entropy": 0.6176188588142395, "entropy_per_token": [0.549547553062439, 0.5073453783988953, 0.04520285502076149, 0.16912707686424255, 0.01202402450144291, 0.23954862356185913, 0.8396081924438477, 0.2154712975025177, 0.11325886845588684, 2.958940029144287, 0.7114628553390503, 0.2869413197040558, 0.42912501096725464, 0.6047168970108032, 0.0036022933200001717, 0.7935174107551575, 0.016773229464888573, 0.8013550043106079, 3.0546956062316895, 0.00011373026063665748], "max_p": 0.8025915026664734, "max_p_per_token": [0.8481981158256531, 0.82770174741745, 0.9939656853675842, 0.9741244316101074, 0.9985443353652954, 0.9374806880950928, 0.6583055853843689, 0.9477127194404602, 0.9836551547050476, 0.10152299702167511, 0.5553207397460938, 0.9182599782943726, 0.929581880569458, 0.8473508954048157, 0.9995976090431213, 0.7791962623596191, 0.9979546070098877, 0.6159758567810059, 0.13739116489887238, 0.9999911785125732], "n_positions_probed": 1, "per_restart_best": [6.578393459320068]}
+{"step": 212, "discrete_loss": 7.756961345672607, "best_sample_loss": 6.879332065582275, "soft_loss": 4.936690330505371, "best_discrete": 6.578393459320068, "best_soft": 4.936690330505371, "best_argmax": 7.70297384262085, "best_sampling": 6.578393459320068, "relax_gap": 0.36357935659181634, "n_match": 7, "g_first_norm": 237.25111389160156, "vocab_size": 50257, "entropy": 0.5993878245353699, "entropy_per_token": [0.5504347681999207, 0.5031852722167969, 0.04454535245895386, 0.1716441512107849, 0.013478565029799938, 0.22668616473674774, 0.827118992805481, 0.21862010657787323, 0.12127295136451721, 2.9674549102783203, 0.713931679725647, 0.3017350435256958, 0.0031204994302242994, 0.5654976963996887, 0.0033157344441860914, 0.7667329907417297, 0.020358411595225334, 0.9103513360023499, 3.058159351348877, 0.00011285494110779837], "max_p": 0.8031107187271118, "max_p_per_token": [0.8482224345207214, 0.8304722905158997, 0.9940628409385681, 0.9735930562019348, 0.9983319640159607, 0.9421316385269165, 0.6680352091789246, 0.9466133117675781, 0.9822244048118591, 0.1005844697356224, 0.5440601706504822, 0.9121468663215637, 0.9997106194496155, 0.8610953092575073, 0.9996337890625, 0.7903591394424438, 0.9974443912506104, 0.5384764671325684, 0.13502365350723267, 0.9999912977218628], "n_positions_probed": 1, "per_restart_best": [6.578393459320068]}
+{"step": 213, "discrete_loss": 7.636100769042969, "best_sample_loss": 6.578393459320068, "soft_loss": 4.882376194000244, "best_discrete": 6.578393459320068, "best_soft": 4.882376194000244, "best_argmax": 7.636100769042969, "best_sampling": 6.578393459320068, "relax_gap": 0.3606192032203693, "n_match": 7, "g_first_norm": 226.718505859375, "vocab_size": 50257, "entropy": 0.5703614950180054, "entropy_per_token": [0.5536589026451111, 0.498306542634964, 0.0440843440592289, 0.17413556575775146, 0.014484856277704239, 0.21309617161750793, 0.8065847754478455, 0.22372277081012726, 0.13020280003547668, 2.964686393737793, 0.7178084850311279, 0.31583836674690247, 0.00400374224409461, 0.0002811821177601814, 0.003038126276805997, 0.7597025036811829, 0.02350660227239132, 0.9138387441635132, 3.046135902404785, 0.00011291520786471665], "max_p": 0.8138663172721863, "max_p_per_token": [0.847412109375, 0.8335651755332947, 0.9941310286521912, 0.9730685353279114, 0.998173713684082, 0.9468852281570435, 0.6843382716178894, 0.9448258280754089, 0.9806079864501953, 0.09199302643537521, 0.5270932912826538, 0.9061363339424133, 0.9996155500411987, 0.9999784231185913, 0.9996683597564697, 0.7932889461517334, 0.9969810843467712, 0.6158831119537354, 0.14368923008441925, 0.9999912977218628], "n_positions_probed": 1, "per_restart_best": [6.578393459320068]}
+{"step": 214, "discrete_loss": 7.636100769042969, "best_sample_loss": 7.443199157714844, "soft_loss": 4.759421348571777, "best_discrete": 6.578393459320068, "best_soft": 4.759421348571777, "best_argmax": 7.636100769042969, "best_sampling": 6.578393459320068, "relax_gap": 0.3767209872522053, "n_match": 7, "g_first_norm": 230.7762908935547, "vocab_size": 50257, "entropy": 0.5779107213020325, "entropy_per_token": [0.5575342178344727, 0.4905778169631958, 0.0433848537504673, 0.17715370655059814, 0.016357313841581345, 0.20253513753414154, 0.7984758615493774, 0.22554107010364532, 0.13878238201141357, 2.95253324508667, 0.7221046686172485, 0.3248278498649597, 0.005034150090068579, 0.0002660407917574048, 0.002869711257517338, 0.7434190511703491, 0.028382549062371254, 1.0762090682983398, 3.0521135330200195, 0.00011234097473789006], "max_p": 0.8038212060928345, "max_p_per_token": [0.8460664749145508, 0.8381243348121643, 0.9942359328269958, 0.9724228978157043, 0.9978792667388916, 0.9504976272583008, 0.6899346709251404, 0.9441363215446472, 0.979015588760376, 0.10096719115972519, 0.5016037225723267, 0.9021730422973633, 0.9995001554489136, 0.9999797344207764, 0.9996918439865112, 0.8000432848930359, 0.9962461590766907, 0.43376386165618896, 0.1301511973142624, 0.9999912977218628], "n_positions_probed": 1, "per_restart_best": [6.578393459320068]}
+{"step": 215, "discrete_loss": 7.701909065246582, "best_sample_loss": 6.206487655639648, "soft_loss": 4.6458635330200195, "best_discrete": 6.206487655639648, "best_soft": 4.6458635330200195, "best_argmax": 7.636100769042969, "best_sampling": 6.206487655639648, "relax_gap": 0.39679065363370675, "n_match": 9, "g_first_norm": 217.58970642089844, "vocab_size": 50257, "entropy": 0.5360799431800842, "entropy_per_token": [0.5624487996101379, 0.4843319058418274, 0.04266373813152313, 0.18225368857383728, 0.017218589782714844, 0.19328634440898895, 0.7747129797935486, 0.22924651205539703, 0.1474604606628418, 2.9290103912353516, 0.7275565266609192, 0.3308444321155548, 0.0064376345835626125, 0.0002532937505748123, 0.002666172105818987, 0.021919164806604385, 0.030644262209534645, 1.0314981937408447, 3.007032871246338, 0.00011297944001853466], "max_p": 0.8222867250442505, "max_p_per_token": [0.8444718718528748, 0.8419660329818726, 0.9943458437919617, 0.9713672399520874, 0.9977268576622009, 0.953570544719696, 0.7078991532325745, 0.942751944065094, 0.977372944355011, 0.11492049694061279, 0.5356703996658325, 0.8994708061218262, 0.9993370175361633, 0.9999808073043823, 0.999716579914093, 0.9971713423728943, 0.9958908557891846, 0.5046355724334717, 0.1674765944480896, 0.9999912977218628], "n_positions_probed": 1, "per_restart_best": [6.206487655639648]}
+{"step": 216, "discrete_loss": 7.701909065246582, "best_sample_loss": 6.546454906463623, "soft_loss": 4.570800304412842, "best_discrete": 6.206487655639648, "best_soft": 4.570800304412842, "best_argmax": 7.636100769042969, "best_sampling": 6.206487655639648, "relax_gap": 0.4065367085366251, "n_match": 9, "g_first_norm": 209.11126708984375, "vocab_size": 50257, "entropy": 0.5344740748405457, "entropy_per_token": [0.5551096200942993, 0.47887831926345825, 0.04198475182056427, 0.18645983934402466, 0.018173731863498688, 0.1866336464881897, 0.761232852935791, 0.22985762357711792, 0.15711647272109985, 2.91276216506958, 0.7346736788749695, 0.32908549904823303, 0.008063157089054585, 0.00024174251302611083, 0.0025525123346596956, 0.023867689073085785, 0.03308998420834541, 1.0027985572814941, 3.0267863273620605, 0.00011316719610476866], "max_p": 0.8223615884780884, "max_p_per_token": [0.8475220799446106, 0.8454275727272034, 0.9944498538970947, 0.9704815745353699, 0.9975548386573792, 0.9557547569274902, 0.7175702452659607, 0.9424090385437012, 0.9754677414894104, 0.12803085148334503, 0.5099186301231384, 0.9002297520637512, 0.9991400241851807, 0.9999816417694092, 0.9997304081916809, 0.9968834519386292, 0.9954912066459656, 0.5269824862480164, 0.14421257376670837, 0.9999911785125732], "n_positions_probed": 1, "per_restart_best": [6.206487655639648]}
+{"step": 217, "discrete_loss": 7.622739315032959, "best_sample_loss": 6.440396308898926, "soft_loss": 4.529106140136719, "best_discrete": 6.206487655639648, "best_soft": 4.529106140136719, "best_argmax": 7.622739315032959, "best_sampling": 6.206487655639648, "relax_gap": 0.40584270916824133, "n_match": 9, "g_first_norm": 205.4838409423828, "vocab_size": 50257, "entropy": 0.48757410049438477, "entropy_per_token": [0.5572071671485901, 0.47225111722946167, 0.04122258350253105, 0.19197042286396027, 0.019465886056423187, 0.17982593178749084, 0.748927116394043, 0.23061300814151764, 0.16728167235851288, 2.896237850189209, 0.7406827211380005, 0.3298790156841278, 0.010124515742063522, 0.0002313316217623651, 0.002444822806864977, 0.025983542203903198, 0.03650897368788719, 0.09025382995605469, 3.010256767272949, 0.00011315653682686388], "max_p": 0.8472877740859985, "max_p_per_token": [0.8466585278511047, 0.8493863344192505, 0.994565486907959, 0.9693145155906677, 0.9973245859146118, 0.9579473733901978, 0.7259119153022766, 0.9420362710952759, 0.973427414894104, 0.14139769971370697, 0.5199571847915649, 0.8998351693153381, 0.9988798499107361, 0.9999825954437256, 0.9997432827949524, 0.9965658783912659, 0.9949370622634888, 0.984489917755127, 0.15340165793895721, 0.9999911785125732], "n_positions_probed": 1, "per_restart_best": [6.206487655639648]}
+{"step": 218, "discrete_loss": 7.692021369934082, "best_sample_loss": 6.179600715637207, "soft_loss": 5.347902774810791, "best_discrete": 6.179600715637207, "best_soft": 4.529106140136719, "best_argmax": 7.622739315032959, "best_sampling": 6.179600715637207, "relax_gap": 0.3047467606220885, "n_match": 8, "g_first_norm": 234.29974365234375, "vocab_size": 50257, "entropy": 0.45914745330810547, "entropy_per_token": [0.5612306594848633, 0.46528956294059753, 0.04048846289515495, 0.19569681584835052, 0.021989954635500908, 0.1713249534368515, 0.7550965547561646, 0.23731033504009247, 0.17099687457084656, 2.869490623474121, 0.7493698000907898, 0.32983270287513733, 0.014155607670545578, 0.00021953528630547225, 0.0024303209502249956, 0.02932528778910637, 0.04284067824482918, 0.09382180869579315, 2.43192195892334, 0.00011676058784360066], "max_p": 0.8593125343322754, "max_p_per_token": [0.8453513383865356, 0.8533681631088257, 0.9946780204772949, 0.9684845209121704, 0.9968956708908081, 0.960654616355896, 0.7215406894683838, 0.9396027326583862, 0.9726125001907349, 0.15078896284103394, 0.49637261033058167, 0.8998264670372009, 0.9983507394790649, 0.9999834299087524, 0.9997450709342957, 0.9960364699363708, 0.993894636631012, 0.9838257431983948, 0.4142480194568634, 0.9999909400939941], "n_positions_probed": 1, "per_restart_best": [6.179600715637207]}
+{"step": 219, "discrete_loss": 7.729046821594238, "best_sample_loss": 7.307483196258545, "soft_loss": 6.463645935058594, "best_discrete": 6.179600715637207, "best_soft": 4.529106140136719, "best_argmax": 7.622739315032959, "best_sampling": 6.179600715637207, "relax_gap": 0.16372017348895238, "n_match": 8, "g_first_norm": 330.0814208984375, "vocab_size": 50257, "entropy": 0.48807650804519653, "entropy_per_token": [0.6012274026870728, 0.45663243532180786, 0.039411697536706924, 0.20358818769454956, 0.02086026966571808, 0.15928031504154205, 0.73417729139328, 0.250724196434021, 0.1697773039340973, 2.8265504837036133, 0.7426820993423462, 0.2778492271900177, 0.022250451147556305, 0.0002108057524310425, 0.0023718001320958138, 0.0339769646525383, 0.0495738759636879, 0.1010514348745346, 3.0692243576049805, 0.00010855032451217994], "max_p": 0.8475834727287292, "max_p_per_token": [0.8305184245109558, 0.8579736948013306, 0.9948373436927795, 0.9667852520942688, 0.9970575571060181, 0.9643396139144897, 0.7347490191459656, 0.9347243309020996, 0.9726216793060303, 0.16659922897815704, 0.5284125804901123, 0.9218010306358337, 0.9972272515296936, 0.9999841451644897, 0.9997521042823792, 0.9952701926231384, 0.9927418231964111, 0.9823294281959534, 0.11395327001810074, 0.9999916553497314], "n_positions_probed": 1, "per_restart_best": [6.179600715637207]}
+{"step": 220, "discrete_loss": 7.729046821594238, "best_sample_loss": 6.203990936279297, "soft_loss": 5.314328193664551, "best_discrete": 6.179600715637207, "best_soft": 4.529106140136719, "best_argmax": 7.622739315032959, "best_sampling": 6.179600715637207, "relax_gap": 0.3124212705224127, "n_match": 8, "g_first_norm": 239.4458770751953, "vocab_size": 50257, "entropy": 0.4924657940864563, "entropy_per_token": [0.663381814956665, 0.4482198655605316, 0.03870366886258125, 0.2103058248758316, 0.023791512474417686, 0.15254859626293182, 0.7374213933944702, 0.25896984338760376, 0.1749720275402069, 2.806351661682129, 0.7432747483253479, 0.28025415539741516, 0.03213924169540405, 0.00020189426140859723, 0.0023237476125359535, 0.03878330439329147, 0.05878306180238724, 0.10611825436353683, 3.072659492492676, 0.000111372210085392], "max_p": 0.8475626111030579, "max_p_per_token": [0.8209629058837891, 0.8625296354293823, 0.994942843914032, 0.965295135974884, 0.9965488314628601, 0.966377854347229, 0.7321972250938416, 0.9316450953483582, 0.97150719165802, 0.17304973304271698, 0.5338522791862488, 0.9208004474639893, 0.9957287907600403, 0.9999849796295166, 0.999757707118988, 0.9944508075714111, 0.9911220073699951, 0.9812491536140442, 0.11925797909498215, 0.9999914169311523], "n_positions_probed": 1, "per_restart_best": [6.179600715637207]}
+{"step": 221, "discrete_loss": 7.729046821594238, "best_sample_loss": 6.172001838684082, "soft_loss": 5.266374111175537, "best_discrete": 6.172001838684082, "best_soft": 4.529106140136719, "best_argmax": 7.622739315032959, "best_sampling": 6.172001838684082, "relax_gap": 0.3186256685026442, "n_match": 8, "g_first_norm": 241.07879638671875, "vocab_size": 50257, "entropy": 0.5000703930854797, "entropy_per_token": [0.6622327566146851, 0.5449361801147461, 0.03797848895192146, 0.21761928498744965, 0.028534183278679848, 0.14669930934906006, 0.7427582144737244, 0.26840078830718994, 0.1811862587928772, 2.789956569671631, 0.7421040534973145, 0.28286755084991455, 0.04600382223725319, 0.00019300678104627877, 0.002283710753545165, 0.04454309120774269, 0.06947027891874313, 0.1121981292963028, 3.081326723098755, 0.00011486246512504295], "max_p": 0.8465143442153931, "max_p_per_token": [0.8222939372062683, 0.8441330194473267, 0.995050847530365, 0.9636539816856384, 0.9957075119018555, 0.9681299924850464, 0.7284678816795349, 0.928061842918396, 0.9701374769210815, 0.17960166931152344, 0.5447856783866882, 0.9197142720222473, 0.9934641718864441, 0.9999856948852539, 0.9997624754905701, 0.9934372305870056, 0.9891681671142578, 0.9798863530158997, 0.11485499143600464, 0.9999910593032837], "n_positions_probed": 1, "per_restart_best": [6.172001838684082]}
+{"step": 222, "discrete_loss": 7.729046821594238, "best_sample_loss": 6.2874860763549805, "soft_loss": 5.241272449493408, "best_discrete": 6.172001838684082, "best_soft": 4.529106140136719, "best_argmax": 7.622739315032959, "best_sampling": 6.172001838684082, "relax_gap": 0.32187337320175363, "n_match": 8, "g_first_norm": 247.15528869628906, "vocab_size": 50257, "entropy": 0.5047487616539001, "entropy_per_token": [0.6591044664382935, 0.5356631278991699, 0.06944908201694489, 0.22611641883850098, 0.034520670771598816, 0.14115644991397858, 0.7485443949699402, 0.27841413021087646, 0.1882481426000595, 2.7776548862457275, 0.735414981842041, 0.2860202491283417, 0.06642941385507584, 0.0001849321706686169, 0.0022445512004196644, 0.05122970789670944, 0.08218139410018921, 0.11899067461490631, 3.093287706375122, 0.00011882439139299095], "max_p": 0.8465504050254822, "max_p_per_token": [0.8242509365081787, 0.8486621975898743, 0.9896929860115051, 0.9617341756820679, 0.9946010112762451, 0.969767689704895, 0.724437952041626, 0.9241766333580017, 0.9685565233230591, 0.1842854917049408, 0.5644923448562622, 0.9183983206748962, 0.9898462891578674, 0.9999862909317017, 0.9997671246528625, 0.9922186136245728, 0.9867513179779053, 0.9782863855361938, 0.11110574007034302, 0.999990701675415], "n_positions_probed": 1, "per_restart_best": [6.172001838684082]}
+{"step": 223, "discrete_loss": 7.729046821594238, "best_sample_loss": 6.193173408508301, "soft_loss": 5.213971138000488, "best_discrete": 6.172001838684082, "best_soft": 4.529106140136719, "best_argmax": 7.622739315032959, "best_sampling": 6.172001838684082, "relax_gap": 0.32540567312477164, "n_match": 8, "g_first_norm": 254.5137481689453, "vocab_size": 50257, "entropy": 0.5083350539207458, "entropy_per_token": [0.6539093255996704, 0.5266343355178833, 0.06854579597711563, 0.23631086945533752, 0.04247576743364334, 0.1359509378671646, 0.753877580165863, 0.28913795948028564, 0.1958804428577423, 2.7669332027435303, 0.7189077138900757, 0.28992822766304016, 0.09624011814594269, 0.00017756418674252927, 0.0022006791550666094, 0.05901765078306198, 0.09730029851198196, 0.12646104395389557, 3.1066882610321045, 0.00012317443906795233], "max_p": 0.8473884463310242, "max_p_per_token": [0.8270121812820435, 0.8529706597328186, 0.9898391962051392, 0.9595447778701782, 0.9930627942085266, 0.9712864756584167, 0.7207636833190918, 0.9199221134185791, 0.9668106436729431, 0.18912872672080994, 0.5973005294799805, 0.9167600870132446, 0.984064519405365, 0.9999868869781494, 0.9997723698616028, 0.9907463788986206, 0.9837557077407837, 0.9764413833618164, 0.10861023515462875, 0.9999903440475464], "n_positions_probed": 1, "per_restart_best": [6.172001838684082]}
+{"step": 224, "discrete_loss": 7.729046821594238, "best_sample_loss": 6.3965373039245605, "soft_loss": 5.178281307220459, "best_discrete": 6.172001838684082, "best_soft": 4.529106140136719, "best_argmax": 7.622739315032959, "best_sampling": 6.172001838684082, "relax_gap": 0.3300232969539242, "n_match": 8, "g_first_norm": 258.7010192871094, "vocab_size": 50257, "entropy": 0.5127253532409668, "entropy_per_token": [0.6493988633155823, 0.5174322724342346, 0.06763441115617752, 0.24573425948619843, 0.053500257432460785, 0.13094311952590942, 0.7586984634399414, 0.30015629529953003, 0.20372304320335388, 2.758697986602783, 0.6942721605300903, 0.29542893171310425, 0.13898849487304688, 0.00017096915689762682, 0.002155024092644453, 0.06812205165624619, 0.11513325572013855, 0.13462954759597778, 3.1195592880249023, 0.00012793237692676485], "max_p": 0.8481775522232056, "max_p_per_token": [0.8295694589614868, 0.8572046160697937, 0.9899855852127075, 0.9573636651039124, 0.9908226728439331, 0.9727265238761902, 0.7175336480140686, 0.9154403209686279, 0.9649685025215149, 0.1933577060699463, 0.6355705857276917, 0.9144451022148132, 0.9748879671096802, 0.9999873638153076, 0.9997777342796326, 0.9889581203460693, 0.9800660610198975, 0.9743298888206482, 0.10656402260065079, 0.9999899864196777], "n_positions_probed": 1, "per_restart_best": [6.172001838684082]}
+{"step": 225, "discrete_loss": 7.729046821594238, "best_sample_loss": 6.15682315826416, "soft_loss": 5.137768745422363, "best_discrete": 6.15682315826416, "best_soft": 4.529106140136719, "best_argmax": 7.622739315032959, "best_sampling": 6.15682315826416, "relax_gap": 0.33526489565726075, "n_match": 8, "g_first_norm": 255.0216827392578, "vocab_size": 50257, "entropy": 0.5189915895462036, "entropy_per_token": [0.6504428386688232, 0.5087392926216125, 0.0666978731751442, 0.25441834330558777, 0.06960850954055786, 0.1261124610900879, 0.7632932662963867, 0.3111514449119568, 0.21135768294334412, 2.7545135021209717, 0.6739113926887512, 0.30313849449157715, 0.1963350921869278, 0.00016468434478156269, 0.0021125124767422676, 0.07876080274581909, 0.13587914407253265, 0.14386288821697235, 3.1291980743408203, 0.00013287001638673246], "max_p": 0.847823441028595, "max_p_per_token": [0.8301981091499329, 0.861092209815979, 0.9901359677314758, 0.9553041458129883, 0.9873330593109131, 0.9740964770317078, 0.7145601511001587, 0.9108388423919678, 0.9631245732307434, 0.19593003392219543, 0.6621463894844055, 0.9111713767051697, 0.9611213207244873, 0.9999879598617554, 0.9997827410697937, 0.986783504486084, 0.9755746126174927, 0.9718399047851562, 0.10545733571052551, 0.9999895095825195], "n_positions_probed": 1, "per_restart_best": [6.15682315826416]}
+{"step": 226, "discrete_loss": 7.740413665771484, "best_sample_loss": 6.123933792114258, "soft_loss": 5.098827362060547, "best_discrete": 6.123933792114258, "best_soft": 4.529106140136719, "best_argmax": 7.622739315032959, "best_sampling": 6.123933792114258, "relax_gap": 0.34127198077179915, "n_match": 9, "g_first_norm": 254.1420135498047, "vocab_size": 50257, "entropy": 0.5447486042976379, "entropy_per_token": [0.6579136848449707, 0.5010360479354858, 0.06573330610990524, 0.2619417905807495, 0.0954013466835022, 0.12157000601291656, 1.111520767211914, 0.3221352994441986, 0.21872259676456451, 2.7539238929748535, 0.6600146889686584, 0.3124149441719055, 0.26990631222724915, 0.00015875583630986512, 0.0020751559641212225, 0.09116888046264648, 0.15963056683540344, 0.15464185178279877, 3.1349246501922607, 0.0001374803832732141], "max_p": 0.832720935344696, "max_p_per_token": [0.8287143707275391, 0.8644868731498718, 0.990291178226471, 0.9534656405448914, 0.9812812209129333, 0.9753682613372803, 0.4418374300003052, 0.9061070084571838, 0.9612972736358643, 0.1974213719367981, 0.6783666610717773, 0.9071674942970276, 0.941196620464325, 0.9999884366989136, 0.9997871518135071, 0.9841383099555969, 0.9701831340789795, 0.968817949295044, 0.10451337695121765, 0.9999891519546509], "n_positions_probed": 1, "per_restart_best": [6.123933792114258]}
+{"step": 227, "discrete_loss": 7.739205360412598, "best_sample_loss": 6.198100566864014, "soft_loss": 5.086710453033447, "best_discrete": 6.123933792114258, "best_soft": 4.529106140136719, "best_argmax": 7.622739315032959, "best_sampling": 6.123933792114258, "relax_gap": 0.34273478785653244, "n_match": 9, "g_first_norm": 255.1150360107422, "vocab_size": 50257, "entropy": 0.5431730151176453, "entropy_per_token": [0.6607272624969482, 0.4970027804374695, 0.06525760889053345, 0.2666321396827698, 0.13876572251319885, 0.11927537620067596, 1.104095458984375, 0.1008271872997284, 0.22797368466854095, 2.752333164215088, 0.6535813808441162, 0.3258748948574066, 0.35701984167099, 0.00015121042088139802, 0.0020558347459882498, 0.10486116260290146, 0.18704451620578766, 0.16843369603157043, 3.1314051151275635, 0.0001414848811691627], "max_p": 0.833467960357666, "max_p_per_token": [0.8289478421211243, 0.8664670586585999, 0.990365743637085, 0.9521994590759277, 0.9699831008911133, 0.9760432839393616, 0.42615270614624023, 0.9792856574058533, 0.9589717388153076, 0.20776647329330444, 0.6849387288093567, 0.9012301564216614, 0.914448082447052, 0.9999890327453613, 0.9997894167900085, 0.9810875058174133, 0.9636650681495667, 0.9647682905197144, 0.10327176004648209, 0.9999887943267822], "n_positions_probed": 1, "per_restart_best": [6.123933792114258]}
+{"step": 228, "discrete_loss": 7.566678047180176, "best_sample_loss": 6.065796852111816, "soft_loss": 5.012358665466309, "best_discrete": 6.065796852111816, "best_soft": 4.529106140136719, "best_argmax": 7.566678047180176, "best_sampling": 6.065796852111816, "relax_gap": 0.3375747409612292, "n_match": 8, "g_first_norm": 270.20166015625, "vocab_size": 50257, "entropy": 0.5625090599060059, "entropy_per_token": [0.6894879341125488, 0.48843830823898315, 0.0639810562133789, 0.26217931509017944, 0.2300550639629364, 0.11866141110658646, 1.1020911931991577, 0.10616900026798248, 0.3261532783508301, 2.7876529693603516, 0.6452879309654236, 0.333139032125473, 0.44175249338150024, 0.0001443078217562288, 0.0020748700480908155, 0.11840736865997314, 0.21642474830150604, 0.187297984957695, 3.130638599395752, 0.00014434110198635608], "max_p": 0.8282719850540161, "max_p_per_token": [0.8207954168319702, 0.8696653842926025, 0.9905949831008911, 0.9530505537986755, 0.9423009157180786, 0.9762921929359436, 0.4442734122276306, 0.9778882265090942, 0.9375505447387695, 0.18434815108776093, 0.6932373046875, 0.8979546427726746, 0.8848861455917358, 0.9999895095825195, 0.9997873902320862, 0.9779360294342041, 0.9562140703201294, 0.9591081142425537, 0.09957773238420486, 0.9999885559082031], "n_positions_probed": 1, "per_restart_best": [6.065796852111816]}
+{"step": 229, "discrete_loss": 7.566678047180176, "best_sample_loss": 5.701696872711182, "soft_loss": 4.9132513999938965, "best_discrete": 5.701696872711182, "best_soft": 4.529106140136719, "best_argmax": 7.566678047180176, "best_sampling": 5.701696872711182, "relax_gap": 0.35067259775577664, "n_match": 8, "g_first_norm": 252.95071411132812, "vocab_size": 50257, "entropy": 0.5791162252426147, "entropy_per_token": [0.7199971079826355, 0.48670220375061035, 0.06258417665958405, 0.260356068611145, 0.34766310453414917, 0.1177724152803421, 1.095831036567688, 0.11208043247461319, 0.34064024686813354, 2.8098387718200684, 0.6383680105209351, 0.340484082698822, 0.529278039932251, 0.0001377263106405735, 0.0020907355938106775, 0.13447964191436768, 0.2484753429889679, 0.20900243520736694, 3.1263961791992188, 0.00014584770542569458], "max_p": 0.824629008769989, "max_p_per_token": [0.8117910623550415, 0.8701848387718201, 0.9908456206321716, 0.9532769322395325, 0.8996591567993164, 0.9766199588775635, 0.47590020298957825, 0.9763135313987732, 0.933693528175354, 0.1885565221309662, 0.6998623609542847, 0.8945948481559753, 0.8504164218902588, 0.9999899864196777, 0.9997856020927429, 0.9740390181541443, 0.9476246237754822, 0.9522215127944946, 0.09721728414297104, 0.9999884366989136], "n_positions_probed": 1, "per_restart_best": [5.701696872711182]}
+{"step": 230, "discrete_loss": 7.471820831298828, "best_sample_loss": 5.927260875701904, "soft_loss": 4.828478813171387, "best_discrete": 5.701696872711182, "best_soft": 4.529106140136719, "best_argmax": 7.471820831298828, "best_sampling": 5.701696872711182, "relax_gap": 0.35377481310241066, "n_match": 8, "g_first_norm": 201.18783569335938, "vocab_size": 50257, "entropy": 0.5938796997070312, "entropy_per_token": [0.7262444496154785, 0.4910588562488556, 0.0615667887032032, 0.2652303874492645, 0.3996380865573883, 0.11680684983730316, 1.086846113204956, 0.11892624944448471, 0.3532521724700928, 2.8117563724517822, 0.6908697485923767, 0.34959739446640015, 0.6092837452888489, 0.000131804816192016, 0.0021214273292571306, 0.1532433032989502, 0.2827465534210205, 0.23395125567913055, 3.1241750717163086, 0.0001467274851165712], "max_p": 0.8213757872581482, "max_p_per_token": [0.8094411492347717, 0.868256688117981, 0.9910305738449097, 0.9519737362861633, 0.8790422677993774, 0.9769763350486755, 0.5077264904975891, 0.9744532108306885, 0.9302006363868713, 0.1915399432182312, 0.6961567401885986, 0.8903540968894958, 0.8147523403167725, 0.9999904632568359, 0.9997822642326355, 0.9692723155021667, 0.9378926753997803, 0.9438376426696777, 0.09484715014696121, 0.999988317489624], "n_positions_probed": 1, "per_restart_best": [5.701696872711182]}
+{"step": 231, "discrete_loss": 7.471820831298828, "best_sample_loss": 5.813141822814941, "soft_loss": 4.774938106536865, "best_discrete": 5.701696872711182, "best_soft": 4.529106140136719, "best_argmax": 7.471820831298828, "best_sampling": 5.701696872711182, "relax_gap": 0.36094049705594494, "n_match": 8, "g_first_norm": 186.6697540283203, "vocab_size": 50257, "entropy": 0.6042978167533875, "entropy_per_token": [0.7315046191215515, 0.495876669883728, 0.06064670905470848, 0.2706799805164337, 0.4344038963317871, 0.11569282412528992, 1.076493740081787, 0.12630242109298706, 0.3652247190475464, 2.8204102516174316, 0.6766566038131714, 0.35966113209724426, 0.67401522397995, 0.00012679147766903043, 0.0021615284495055676, 0.17461520433425903, 0.3190845847129822, 0.2617551386356354, 3.1204957962036133, 0.00014750029367860407], "max_p": 0.8186527490615845, "max_p_per_token": [0.8072377443313599, 0.8660270571708679, 0.991199791431427, 0.9505462646484375, 0.865541398525238, 0.9773756265640259, 0.5335684418678284, 0.972405195236206, 0.926790177822113, 0.19147589802742004, 0.7052241563796997, 0.8855136632919312, 0.7824751734733582, 0.9999908208847046, 0.999777615070343, 0.9635653495788574, 0.9269534349441528, 0.9338885545730591, 0.09351079165935516, 0.9999881982803345], "n_positions_probed": 1, "per_restart_best": [5.701696872711182]}
+{"step": 232, "discrete_loss": 7.471820831298828, "best_sample_loss": 6.059214115142822, "soft_loss": 4.730404376983643, "best_discrete": 5.701696872711182, "best_soft": 4.529106140136719, "best_argmax": 7.471820831298828, "best_sampling": 5.701696872711182, "relax_gap": 0.36690072155258635, "n_match": 8, "g_first_norm": 175.6945037841797, "vocab_size": 50257, "entropy": 0.6265599131584167, "entropy_per_token": [0.7379530668258667, 0.5007489919662476, 0.0596994049847126, 0.2765108346939087, 0.45821207761764526, 0.11462759971618652, 1.0700256824493408, 0.1339854896068573, 0.37679189443588257, 2.8314971923828125, 0.6648286581039429, 0.3707561790943146, 0.9680155515670776, 0.00012251842417754233, 0.0022095968015491962, 0.198654904961586, 0.3573240339756012, 0.29250580072402954, 3.11657977104187, 0.0001482060324633494], "max_p": 0.8121882677078247, "max_p_per_token": [0.8046116232872009, 0.8636842966079712, 0.9913731217384338, 0.9490278363227844, 0.8574541807174683, 0.9777668714523315, 0.5482571125030518, 0.9702242016792297, 0.9234079122543335, 0.19107800722122192, 0.712433397769928, 0.8800988793373108, 0.6891096234321594, 0.9999911785125732, 0.9997721314430237, 0.9568032622337341, 0.9147475361824036, 0.9221158027648926, 0.09181863814592361, 0.9999881982803345], "n_positions_probed": 1, "per_restart_best": [5.701696872711182]}
+{"step": 233, "discrete_loss": 7.471820831298828, "best_sample_loss": 6.7289628982543945, "soft_loss": 4.672312259674072, "best_discrete": 5.701696872711182, "best_soft": 4.529106140136719, "best_argmax": 7.471820831298828, "best_sampling": 5.701696872711182, "relax_gap": 0.37467554895024385, "n_match": 8, "g_first_norm": 163.9420166015625, "vocab_size": 50257, "entropy": 0.6366434097290039, "entropy_per_token": [0.7489159107208252, 0.5067625045776367, 0.058818139135837555, 0.28181493282318115, 0.4688189923763275, 0.11388442665338516, 1.0667872428894043, 0.14198212325572968, 0.387803852558136, 2.8448779582977295, 0.6552128791809082, 0.3824193775653839, 1.0095250606536865, 0.00012524641351774335, 0.002255320316180587, 0.22453981637954712, 0.3970189690589905, 0.32744449377059937, 3.1137123107910156, 0.00014835337060503662], "max_p": 0.8088216781616211, "max_p_per_token": [0.8004379868507385, 0.8607527613639832, 0.9915353059768677, 0.9476577043533325, 0.856385350227356, 0.9780796766281128, 0.5566000938415527, 0.9679028391838074, 0.9200977087020874, 0.18984775245189667, 0.7179620862007141, 0.8742603063583374, 0.6667383313179016, 0.9999910593032837, 0.9997668862342834, 0.9491288661956787, 0.9012861251831055, 0.9076871871948242, 0.09032630175352097, 0.9999881982803345], "n_positions_probed": 1, "per_restart_best": [5.701696872711182]}
+{"step": 234, "discrete_loss": 7.531070709228516, "best_sample_loss": 6.526711940765381, "soft_loss": 4.640649318695068, "best_discrete": 5.701696872711182, "best_soft": 4.529106140136719, "best_argmax": 7.471820831298828, "best_sampling": 5.701696872711182, "relax_gap": 0.38379952892907343, "n_match": 8, "g_first_norm": 162.38723754882812, "vocab_size": 50257, "entropy": 0.647047221660614, "entropy_per_token": [0.7609938383102417, 0.5132145881652832, 0.057872358709573746, 0.2875650227069855, 0.48011475801467896, 0.11338240653276443, 1.064787745475769, 0.15026767551898956, 0.3990814983844757, 2.858424186706543, 0.6466407775878906, 0.3925403654575348, 1.0474481582641602, 0.00012107689690310508, 0.0023051861207932234, 0.25260791182518005, 0.43780258297920227, 0.365691602230072, 3.1099343299865723, 0.00014860219380352646], "max_p": 0.8051132559776306, "max_p_per_token": [0.7957998514175415, 0.8576235175132751, 0.9917076230049133, 0.9461668133735657, 0.8555509448051453, 0.9783356785774231, 0.561791718006134, 0.9654427170753479, 0.9166414141654968, 0.18918262422084808, 0.7227679491043091, 0.8690604567527771, 0.6454384922981262, 0.9999914169311523, 0.9997615218162537, 0.940346360206604, 0.8865868449211121, 0.8905314803123474, 0.08954857289791107, 0.9999881982803345], "n_positions_probed": 1, "per_restart_best": [5.701696872711182]}
+{"step": 235, "discrete_loss": 7.531070709228516, "best_sample_loss": 6.534959316253662, "soft_loss": 4.6089677810668945, "best_discrete": 5.701696872711182, "best_soft": 4.529106140136719, "best_argmax": 7.471820831298828, "best_sampling": 5.701696872711182, "relax_gap": 0.38800630627208144, "n_match": 8, "g_first_norm": 161.6638946533203, "vocab_size": 50257, "entropy": 0.6578999757766724, "entropy_per_token": [0.7732067108154297, 0.5200986862182617, 0.05688783526420593, 0.2936975061893463, 0.4927125871181488, 0.11316990852355957, 1.0631418228149414, 0.1588972508907318, 0.41058409214019775, 2.873426914215088, 0.6389160752296448, 0.4009518325328827, 1.0814999341964722, 0.00011702576011884958, 0.002350342459976673, 0.28625229001045227, 0.4782639741897583, 0.4074181020259857, 3.1062569618225098, 0.00014905775606166571], "max_p": 0.8011198043823242, "max_p_per_token": [0.7910330295562744, 0.854289710521698, 0.9918854236602783, 0.944570004940033, 0.8546896576881409, 0.9785231351852417, 0.5654413104057312, 0.9628211259841919, 0.9130468368530273, 0.18804533779621124, 0.7270073294639587, 0.8646407723426819, 0.6256017684936523, 0.999991774559021, 0.9997562766075134, 0.9300925731658936, 0.8710803985595703, 0.8700169920921326, 0.089875228703022, 0.9999880790710449], "n_positions_probed": 1, "per_restart_best": [5.701696872711182]}
+{"step": 236, "discrete_loss": 7.531070709228516, "best_sample_loss": 6.523810863494873, "soft_loss": 4.576471328735352, "best_discrete": 5.701696872711182, "best_soft": 4.529106140136719, "best_argmax": 7.471820831298828, "best_sampling": 5.701696872711182, "relax_gap": 0.3923212906330332, "n_match": 8, "g_first_norm": 161.55775451660156, "vocab_size": 50257, "entropy": 0.6667044758796692, "entropy_per_token": [0.785658061504364, 0.5274158716201782, 0.05587383359670639, 0.30021512508392334, 0.5073005557060242, 0.11323115229606628, 1.0614173412322998, 0.1678953468799591, 0.4223426580429077, 2.8890762329101562, 0.6318886280059814, 0.4076612591743469, 1.1114836931228638, 0.00011318581528030336, 0.0023946580477058887, 0.3170120120048523, 0.47781920433044434, 0.4524710476398468, 3.1026697158813477, 0.00014946601004339755], "max_p": 0.7973426580429077, "max_p_per_token": [0.7860708832740784, 0.8507485389709473, 0.9920674562454224, 0.9428640604019165, 0.8535134196281433, 0.9786471128463745, 0.5682733058929443, 0.9600233435630798, 0.9093009829521179, 0.18668951094150543, 0.73079514503479, 0.8610485792160034, 0.6075326204299927, 0.9999920129776001, 0.9997511506080627, 0.9193593263626099, 0.8645749688148499, 0.84544438123703, 0.09016724675893784, 0.9999880790710449], "n_positions_probed": 1, "per_restart_best": [5.701696872711182]}
+{"step": 237, "discrete_loss": 7.531070709228516, "best_sample_loss": 5.8779826164245605, "soft_loss": 4.544266700744629, "best_discrete": 5.701696872711182, "best_soft": 4.529106140136719, "best_argmax": 7.471820831298828, "best_sampling": 5.701696872711182, "relax_gap": 0.39659752561131584, "n_match": 8, "g_first_norm": 162.06637573242188, "vocab_size": 50257, "entropy": 0.6547278761863708, "entropy_per_token": [0.7978993058204651, 0.5351204872131348, 0.05484173074364662, 0.3070540130138397, 0.524454653263092, 0.11358129978179932, 1.0593981742858887, 0.177333801984787, 0.4343295097351074, 2.9056081771850586, 0.6255477666854858, 0.4127228260040283, 1.137155294418335, 0.00010938671766780317, 0.0024361899122595787, 0.34781983494758606, 0.5139227509498596, 0.04566499963402748, 3.0994067192077637, 0.00014993180229794234], "max_p": 0.8016492128372192, "max_p_per_token": [0.7810602784156799, 0.8470120429992676, 0.9922513365745544, 0.9410600066184998, 0.8517802357673645, 0.9787050485610962, 0.5706315040588379, 0.9570179581642151, 0.9054107666015625, 0.18475571274757385, 0.7341338396072388, 0.8582969307899475, 0.5915163159370422, 0.9999922513961792, 0.9997463822364807, 0.9080435633659363, 0.8487581610679626, 0.9923878908157349, 0.09043645858764648, 0.9999880790710449], "n_positions_probed": 1, "per_restart_best": [5.701696872711182]}
+{"step": 238, "discrete_loss": 7.521447658538818, "best_sample_loss": 5.701696872711182, "soft_loss": 4.579272747039795, "best_discrete": 5.701696872711182, "best_soft": 4.529106140136719, "best_argmax": 7.471820831298828, "best_sampling": 5.701696872711182, "relax_gap": 0.39117136023128235, "n_match": 8, "g_first_norm": 161.712646484375, "vocab_size": 50257, "entropy": 0.635393500328064, "entropy_per_token": [0.8042535781860352, 0.5448279976844788, 0.053724367171525955, 0.3137919306755066, 0.5416496992111206, 0.11420051753520966, 1.055212378501892, 0.18719394505023956, 0.4445679187774658, 2.920243978500366, 0.6190785765647888, 0.4159928858280182, 1.1571691036224365, 0.00010571435268502682, 0.002477998612448573, 0.3795323073863983, 0.5507479310035706, 0.05333602800965309, 2.5496132373809814, 0.0001494426978752017], "max_p": 0.81061190366745, "max_p_per_token": [0.778249204158783, 0.8422731757164001, 0.9924474954605103, 0.9392442107200623, 0.8503812551498413, 0.9787042140960693, 0.5741652846336365, 0.9538008570671082, 0.9019380211830139, 0.1843290776014328, 0.7377792000770569, 0.8565038442611694, 0.5791529417037964, 0.9999924898147583, 0.9997414946556091, 0.8956813216209412, 0.8317697048187256, 0.9907880425453186, 0.3253074586391449, 0.9999880790710449], "n_positions_probed": 1, "per_restart_best": [5.701696872711182]}
+{"step": 239, "discrete_loss": 7.567149639129639, "best_sample_loss": 6.481212139129639, "soft_loss": 5.169798374176025, "best_discrete": 5.701696872711182, "best_soft": 4.529106140136719, "best_argmax": 7.471820831298828, "best_sampling": 5.701696872711182, "relax_gap": 0.3168103419756548, "n_match": 8, "g_first_norm": 220.5744171142578, "vocab_size": 50257, "entropy": 0.6683842539787292, "entropy_per_token": [0.8907806873321533, 0.5580945611000061, 0.05221758037805557, 0.31132155656814575, 0.5739396810531616, 0.1144528016448021, 1.0633491277694702, 0.19742116332054138, 0.443072110414505, 2.888617992401123, 0.6061126589775085, 0.4169054627418518, 1.1897423267364502, 9.730827150633559e-05, 0.0024291127920150757, 0.40819936990737915, 0.5710256099700928, 0.06237607076764107, 3.0173773765563965, 0.00015405882732011378], "max_p": 0.797162652015686, "max_p_per_token": [0.7442660331726074, 0.836277961730957, 0.9927070736885071, 0.9399908185005188, 0.8421444892883301, 0.9787710309028625, 0.5657008290290833, 0.9503800272941589, 0.9014104604721069, 0.2114700824022293, 0.7471115589141846, 0.856128454208374, 0.5604138374328613, 0.9999932050704956, 0.9997473359107971, 0.8838913440704346, 0.8215029239654541, 0.9888100028038025, 0.12254734337329865, 0.9999878406524658], "n_positions_probed": 1, "per_restart_best": [5.701696872711182]}
+{"step": 240, "discrete_loss": 7.521447658538818, "best_sample_loss": 5.714433193206787, "soft_loss": 4.63585901260376, "best_discrete": 5.701696872711182, "best_soft": 4.529106140136719, "best_argmax": 7.471820831298828, "best_sampling": 5.701696872711182, "relax_gap": 0.38364803917224066, "n_match": 8, "g_first_norm": 164.63510131835938, "vocab_size": 50257, "entropy": 0.6750039458274841, "entropy_per_token": [0.7851861119270325, 0.5679447650909424, 0.05125611275434494, 0.31800514459609985, 0.5891494750976562, 0.1175193190574646, 1.0575278997421265, 0.20752036571502686, 0.45412296056747437, 2.957521438598633, 0.5994215607643127, 0.41826969385147095, 1.2035220861434937, 9.2905800556764e-05, 0.002473237691447139, 0.4318876266479492, 0.5974551439285278, 0.07268751412630081, 3.068361759185791, 0.00015414021618198603], "max_p": 0.7944398522377014, "max_p_per_token": [0.8015843033790588, 0.8311945796012878, 0.9928770661354065, 0.938271701335907, 0.8409403562545776, 0.9781848192214966, 0.5693658590316772, 0.9469174146652222, 0.8976956605911255, 0.17995460331439972, 0.7510969042778015, 0.8553885221481323, 0.55076003074646, 0.9999935626983643, 0.9997422099113464, 0.8738863468170166, 0.808331310749054, 0.9864622950553894, 0.08616071194410324, 0.9999878406524658], "n_positions_probed": 1, "per_restart_best": [5.701696872711182]}
+{"step": 241, "discrete_loss": 7.471820831298828, "best_sample_loss": 5.66573429107666, "soft_loss": 4.557188510894775, "best_discrete": 5.66573429107666, "best_soft": 4.529106140136719, "best_argmax": 7.471820831298828, "best_sampling": 5.66573429107666, "relax_gap": 0.3900832723658072, "n_match": 8, "g_first_norm": 159.88375854492188, "vocab_size": 50257, "entropy": 0.6825156807899475, "entropy_per_token": [0.8067817687988281, 0.5750985145568848, 0.050154320895671844, 0.3266674876213074, 0.6065998077392578, 0.11874396353960037, 1.0520751476287842, 0.2183486372232437, 0.46529296040534973, 2.961049795150757, 0.5944952368736267, 0.41768044233322144, 1.213687539100647, 8.949616312747821e-05, 0.0024984250776469707, 0.45557701587677, 0.6200281977653503, 0.08479052037000656, 3.080500602722168, 0.00015364370483439416], "max_p": 0.79256671667099, "max_p_per_token": [0.7934744358062744, 0.8276371359825134, 0.9930676221847534, 0.9359698295593262, 0.8393678069114685, 0.9780463576316833, 0.5724120736122131, 0.9431105256080627, 0.8937748670578003, 0.18659082055091858, 0.7537744641304016, 0.8557187914848328, 0.5432863235473633, 0.9999938011169434, 0.9997392296791077, 0.8636094927787781, 0.7964403033256531, 0.9835798144340515, 0.09175273776054382, 0.9999877214431763], "n_positions_probed": 1, "per_restart_best": [5.66573429107666]}
+{"step": 242, "discrete_loss": 7.471820831298828, "best_sample_loss": 5.649564266204834, "soft_loss": 4.519761085510254, "best_discrete": 5.649564266204834, "best_soft": 4.519761085510254, "best_argmax": 7.471820831298828, "best_sampling": 5.649564266204834, "relax_gap": 0.39509241621836605, "n_match": 8, "g_first_norm": 160.96534729003906, "vocab_size": 50257, "entropy": 0.6889406442642212, "entropy_per_token": [0.8185415267944336, 0.5825321674346924, 0.03357064723968506, 0.3354600965976715, 0.6271824836730957, 0.12032316625118256, 1.0462300777435303, 0.23020711541175842, 0.476057231426239, 2.980290412902832, 0.5905500650405884, 0.4162592887878418, 1.2195606231689453, 8.623427856946364e-05, 0.002525835996493697, 0.47634440660476685, 0.6384423971176147, 0.09883585572242737, 3.085660457611084, 0.00015260186046361923], "max_p": 0.7903662919998169, "max_p_per_token": [0.7888285517692566, 0.8235575556755066, 0.9959094524383545, 0.933588981628418, 0.8371081352233887, 0.9778297543525696, 0.5752543210983276, 0.9388282895088196, 0.8899410367012024, 0.18257668614387512, 0.7557632923126221, 0.856507420539856, 0.5382246971130371, 0.9999940395355225, 0.9997360110282898, 0.854449450969696, 0.7865764498710632, 0.9800688028335571, 0.09259434789419174, 0.9999878406524658], "n_positions_probed": 1, "per_restart_best": [5.649564266204834]}
+{"step": 243, "discrete_loss": 7.471820831298828, "best_sample_loss": 5.640870571136475, "soft_loss": 4.492820739746094, "best_discrete": 5.640870571136475, "best_soft": 4.492820739746094, "best_argmax": 7.471820831298828, "best_sampling": 5.640870571136475, "relax_gap": 0.3986980093358173, "n_match": 7, "g_first_norm": 161.97645568847656, "vocab_size": 50257, "entropy": 0.6957339644432068, "entropy_per_token": [0.8304398059844971, 0.5905264616012573, 0.03300948813557625, 0.34506016969680786, 0.650242805480957, 0.12187736481428146, 1.0399154424667358, 0.2426908016204834, 0.48651695251464844, 2.995819091796875, 0.5870476961135864, 0.4147278368473053, 1.2228021621704102, 8.322572102770209e-05, 0.002545673865824938, 0.49479344487190247, 0.6532571911811829, 0.11502932012081146, 3.088141918182373, 0.00015150359831750393], "max_p": 0.7880704998970032, "max_p_per_token": [0.7840365767478943, 0.8191475868225098, 0.9959914088249207, 0.9310669898986816, 0.8342120051383972, 0.9776236414909363, 0.5781016945838928, 0.9341901540756226, 0.8861420750617981, 0.17970533668994904, 0.7574717402458191, 0.8573554158210754, 0.5344809293746948, 0.999994158744812, 0.9997338652610779, 0.8462642431259155, 0.7784744501113892, 0.97580885887146, 0.09162081032991409, 0.9999879598617554], "n_positions_probed": 1, "per_restart_best": [5.640870571136475]}
+{"step": 244, "discrete_loss": 7.646884441375732, "best_sample_loss": 6.079159736633301, "soft_loss": 4.46869421005249, "best_discrete": 5.640870571136475, "best_soft": 4.46869421005249, "best_argmax": 7.471820831298828, "best_sampling": 5.640870571136475, "relax_gap": 0.4156189694886329, "n_match": 6, "g_first_norm": 162.8316192626953, "vocab_size": 50257, "entropy": 0.7178394198417664, "entropy_per_token": [0.8421006202697754, 0.5989737510681152, 0.032454755157232285, 0.3539932370185852, 0.9860792756080627, 0.12348932772874832, 1.0333948135375977, 0.2557823657989502, 0.496576726436615, 3.0105395317077637, 0.5838637351989746, 0.4133407473564148, 1.2241847515106201, 8.020361565286294e-05, 0.0025594173930585384, 0.5109219551086426, 0.6650218963623047, 0.1337186098098755, 3.0895633697509766, 0.00015040770813357085], "max_p": 0.7683685421943665, "max_p_per_token": [0.7792611718177795, 0.8144373893737793, 0.9960721731185913, 0.9285584092140198, 0.48290663957595825, 0.9774073958396912, 0.5807620286941528, 0.929180383682251, 0.8824177384376526, 0.17618504166603088, 0.7589956521987915, 0.8581238985061646, 0.5316389799118042, 0.9999943971633911, 0.9997323155403137, 0.8391595482826233, 0.7719717621803284, 0.970619261264801, 0.08995801210403442, 0.9999880790710449], "n_positions_probed": 1, "per_restart_best": [5.640870571136475]}
+{"step": 245, "discrete_loss": 7.471820831298828, "best_sample_loss": 5.858363628387451, "soft_loss": 4.490066051483154, "best_discrete": 5.640870571136475, "best_soft": 4.46869421005249, "best_argmax": 7.471820831298828, "best_sampling": 5.640870571136475, "relax_gap": 0.399066686305613, "n_match": 7, "g_first_norm": 156.36082458496094, "vocab_size": 50257, "entropy": 0.7275623679161072, "entropy_per_token": [0.8709648847579956, 0.6052701473236084, 0.03143531456589699, 0.3568973243236542, 1.0081133842468262, 0.12533216178417206, 1.0349807739257812, 0.2704373002052307, 0.5054517388343811, 3.0265040397644043, 0.5819689035415649, 0.42636755108833313, 1.241891622543335, 7.830200047465041e-05, 0.002659379504621029, 0.5304713249206543, 0.691735029220581, 0.15528103709220886, 3.085254430770874, 0.00015232106670737267], "max_p": 0.7649622559547424, "max_p_per_token": [0.7677836418151855, 0.8095044493675232, 0.9962180256843567, 0.9275374412536621, 0.4996914863586426, 0.977181077003479, 0.5772393345832825, 0.9233911633491516, 0.8794935941696167, 0.17049452662467957, 0.7595183253288269, 0.8509394526481628, 0.5198343396186829, 0.9999945163726807, 0.9997205138206482, 0.8302491903305054, 0.7568966746330261, 0.9642849564552307, 0.08928379416465759, 0.9999879598617554], "n_positions_probed": 1, "per_restart_best": [5.640870571136475]}
+{"step": 246, "discrete_loss": 7.531070709228516, "best_sample_loss": 5.796861171722412, "soft_loss": 4.426705837249756, "best_discrete": 5.640870571136475, "best_soft": 4.426705837249756, "best_argmax": 7.471820831298828, "best_sampling": 5.640870571136475, "relax_gap": 0.41220763844040065, "n_match": 7, "g_first_norm": 149.72811889648438, "vocab_size": 50257, "entropy": 0.7338477969169617, "entropy_per_token": [0.8891408443450928, 0.6124420166015625, 0.03049248829483986, 0.35988011956214905, 1.0023328065872192, 0.1258612722158432, 1.0338574647903442, 0.2850843071937561, 0.5148557424545288, 3.037893056869507, 0.5798126459121704, 0.43270865082740784, 1.2516423463821411, 7.595161878271028e-05, 0.0027320755179971457, 0.5480204820632935, 0.7096431255340576, 0.179117813706398, 3.0812082290649414, 0.00015388880274258554], "max_p": 0.7642378211021423, "max_p_per_token": [0.7599967122077942, 0.8042341470718384, 0.996351957321167, 0.92649245262146, 0.5445600748062134, 0.9772266149520874, 0.5770127177238464, 0.917408287525177, 0.8762720823287964, 0.17128126323223114, 0.7604140043258667, 0.8473812937736511, 0.5120400786399841, 0.9999947547912598, 0.9997119307518005, 0.8224052786827087, 0.7467237710952759, 0.9568439722061157, 0.08841633796691895, 0.9999878406524658], "n_positions_probed": 1, "per_restart_best": [5.640870571136475]}
+{"step": 247, "discrete_loss": 7.531070709228516, "best_sample_loss": 5.764492511749268, "soft_loss": 4.385161399841309, "best_discrete": 5.640870571136475, "best_soft": 4.385161399841309, "best_argmax": 7.471820831298828, "best_sampling": 5.640870571136475, "relax_gap": 0.4177240436120503, "n_match": 7, "g_first_norm": 152.12612915039062, "vocab_size": 50257, "entropy": 0.7388679385185242, "entropy_per_token": [0.903362512588501, 0.6205540895462036, 0.029688384383916855, 0.3624962568283081, 0.9866849184036255, 0.12624473869800568, 1.0294053554534912, 0.30037662386894226, 0.524660587310791, 3.0511016845703125, 0.5775457620620728, 0.43463289737701416, 1.2565720081329346, 7.335797999985516e-05, 0.0027845643926411867, 0.5641268491744995, 0.721127450466156, 0.205993190407753, 3.07977294921875, 0.00015531686949543655], "max_p": 0.7634763121604919, "max_p_per_token": [0.7535883784294128, 0.7985473871231079, 0.996465802192688, 0.9255748391151428, 0.578348696231842, 0.9773144125938416, 0.5777583718299866, 0.9109453558921814, 0.8727870583534241, 0.16876903176307678, 0.761464536190033, 0.8463255763053894, 0.5064579248428345, 0.9999949932098389, 0.9997057318687439, 0.8153321146965027, 0.7402946949005127, 0.9478996992111206, 0.0919627845287323, 0.9999877214431763], "n_positions_probed": 1, "per_restart_best": [5.640870571136475]}
+{"step": 248, "discrete_loss": 7.531070709228516, "best_sample_loss": 5.694869518280029, "soft_loss": 4.352059364318848, "best_discrete": 5.640870571136475, "best_soft": 4.352059364318848, "best_argmax": 7.471820831298828, "best_sampling": 5.640870571136475, "relax_gap": 0.42211943927363904, "n_match": 7, "g_first_norm": 154.09156799316406, "vocab_size": 50257, "entropy": 0.7441410422325134, "entropy_per_token": [0.9156585931777954, 0.6291743516921997, 0.02895362675189972, 0.3656842112541199, 0.9704185128211975, 0.12697364389896393, 1.0242449045181274, 0.3162557780742645, 0.5469704270362854, 3.0638911724090576, 0.5751169323921204, 0.4341272711753845, 1.2579870223999023, 7.076394831528887e-05, 0.002819676883518696, 0.5789942741394043, 0.7282453775405884, 0.23647888004779816, 3.0805981159210205, 0.00015671095752622932], "max_p": 0.7624022960662842, "max_p_per_token": [0.7478459477424622, 0.7926437258720398, 0.9965692758560181, 0.9245162010192871, 0.6044126152992249, 0.9773239493370056, 0.5788999795913696, 0.9039912819862366, 0.8674712777137756, 0.165411114692688, 0.7626875638961792, 0.8466663956642151, 0.5024848580360413, 0.9999951124191284, 0.9997015595436096, 0.8089507222175598, 0.7365175485610962, 0.9370284080505371, 0.09494128823280334, 0.9999876022338867], "n_positions_probed": 1, "per_restart_best": [5.640870571136475]}
+{"step": 249, "discrete_loss": 7.262202739715576, "best_sample_loss": 6.093791484832764, "soft_loss": 4.322813034057617, "best_discrete": 5.640870571136475, "best_soft": 4.322813034057617, "best_argmax": 7.262202739715576, "best_sampling": 5.640870571136475, "relax_gap": 0.40475181029896173, "n_match": 7, "g_first_norm": 155.81471252441406, "vocab_size": 50257, "entropy": 0.6808211207389832, "entropy_per_token": [0.9263989925384521, 0.6378862261772156, 0.028272980824112892, 0.36966612935066223, 0.957442045211792, 0.12819746136665344, 1.0189130306243896, 0.33264902234077454, 0.5560451745986938, 1.7158229351043701, 0.5726057887077332, 0.4324905276298523, 1.2569074630737305, 6.788421887904406e-05, 0.002839331980794668, 0.592778205871582, 0.7328277230262756, 0.27091047167778015, 3.0835418701171875, 0.00015811517369002104], "max_p": 0.775819718837738, "max_p_per_token": [0.7426679134368896, 0.7867234349250793, 0.9966644644737244, 0.9232472777366638, 0.6225889325141907, 0.9772199988365173, 0.5800561308860779, 0.8965415954589844, 0.8639118075370789, 0.45730599761009216, 0.7640113234519958, 0.8476552367210388, 0.49960535764694214, 0.9999954700469971, 0.9996993541717529, 0.8032084107398987, 0.7343113422393799, 0.9237870573997498, 0.09720570594072342, 0.9999874830245972], "n_positions_probed": 1, "per_restart_best": [5.640870571136475]}
+{"step": 250, "discrete_loss": 7.268826484680176, "best_sample_loss": 5.869269847869873, "soft_loss": 5.037554740905762, "best_discrete": 5.640870571136475, "best_soft": 4.322813034057617, "best_argmax": 7.262202739715576, "best_sampling": 5.640870571136475, "relax_gap": 0.306964507747854, "n_match": 7, "g_first_norm": 153.24774169921875, "vocab_size": 50257, "entropy": 0.7040264010429382, "entropy_per_token": [0.9415385127067566, 0.6575585603713989, 0.028357943519949913, 0.3792164623737335, 0.9580402374267578, 0.12815146148204803, 1.020337462425232, 0.3442118167877197, 0.5862263441085815, 1.9992082118988037, 0.587143063545227, 0.4538302421569824, 1.2705557346343994, 6.481393938884139e-05, 0.0027808593586087227, 0.5988078117370605, 0.7312321662902832, 0.31173405051231384, 3.0813705921173096, 0.00016238506941590458], "max_p": 0.7650238275527954, "max_p_per_token": [0.7364687919616699, 0.7760547399520874, 0.9966554641723633, 0.9203815460205078, 0.6281188130378723, 0.977374255657196, 0.5729317665100098, 0.8911147713661194, 0.8528878092765808, 0.3333820700645447, 0.7545379996299744, 0.8353171348571777, 0.4910319447517395, 0.9999955892562866, 0.9997065663337708, 0.8041352033615112, 0.7346526980400085, 0.9066846966743469, 0.08905624598264694, 0.999987006187439], "n_positions_probed": 1, "per_restart_best": [5.640870571136475]}
+{"step": 251, "discrete_loss": 7.30139684677124, "best_sample_loss": 5.792349338531494, "soft_loss": 4.814356803894043, "best_discrete": 5.640870571136475, "best_soft": 4.322813034057617, "best_argmax": 7.262202739715576, "best_sampling": 5.640870571136475, "relax_gap": 0.3406252385770531, "n_match": 7, "g_first_norm": 158.09527587890625, "vocab_size": 50257, "entropy": 0.7273024320602417, "entropy_per_token": [0.9465325474739075, 0.6793539524078369, 0.02847491018474102, 0.3878394067287445, 0.9642161130905151, 0.12783867120742798, 1.0172762870788574, 0.35930734872817993, 0.6088106036186218, 2.3002419471740723, 0.5923239588737488, 0.4645659625530243, 1.280552864074707, 6.124462379375473e-05, 0.0027666017413139343, 0.6099939346313477, 0.7347127199172974, 0.35933616757392883, 3.081674575805664, 0.00016787480853963643], "max_p": 0.7563080191612244, "max_p_per_token": [0.7343091368675232, 0.763843297958374, 0.9966425895690918, 0.9176883697509766, 0.6244276762008667, 0.9775910377502441, 0.5697855949401855, 0.8838057518005371, 0.844175398349762, 0.24570411443710327, 0.7494959831237793, 0.8291295766830444, 0.48329097032546997, 0.9999958276748657, 0.9997084736824036, 0.8015258312225342, 0.7323340773582458, 0.8846215605735779, 0.08809719979763031, 0.9999866485595703], "n_positions_probed": 1, "per_restart_best": [5.640870571136475]}
+{"step": 252, "discrete_loss": 7.320648193359375, "best_sample_loss": 6.686293125152588, "soft_loss": 4.565499305725098, "best_discrete": 5.640870571136475, "best_soft": 4.322813034057617, "best_argmax": 7.262202739715576, "best_sampling": 5.640870571136475, "relax_gap": 0.37635313361096867, "n_match": 7, "g_first_norm": 164.05491638183594, "vocab_size": 50257, "entropy": 0.7473007440567017, "entropy_per_token": [0.9523963928222656, 0.6966677904129028, 0.02859603613615036, 0.3984632194042206, 0.9728749990463257, 0.1273895502090454, 1.0134913921356201, 0.37756550312042236, 0.6166431307792664, 2.543304920196533, 0.5931461453437805, 0.4678802490234375, 1.2860422134399414, 5.785049870610237e-05, 0.002769751474261284, 0.6251782178878784, 0.7437452673912048, 0.4112711250782013, 3.088358163833618, 0.0001729822251945734], "max_p": 0.7499820590019226, "max_p_per_token": [0.7314420938491821, 0.7539982795715332, 0.9966288208961487, 0.9143394231796265, 0.6176643967628479, 0.9778393507003784, 0.5693444013595581, 0.8746089339256287, 0.8404901027679443, 0.20243796706199646, 0.7479358911514282, 0.8272110819816589, 0.47666651010513306, 0.9999960660934448, 0.9997082352638245, 0.7960904240608215, 0.7264314293861389, 0.857542097568512, 0.08927926421165466, 0.9999861717224121], "n_positions_probed": 1, "per_restart_best": [5.640870571136475]}
+{"step": 253, "discrete_loss": 7.320648193359375, "best_sample_loss": 6.805624485015869, "soft_loss": 4.3642683029174805, "best_discrete": 5.640870571136475, "best_soft": 4.322813034057617, "best_argmax": 7.262202739715576, "best_sampling": 5.640870571136475, "relax_gap": 0.40384127366257716, "n_match": 7, "g_first_norm": 155.42376708984375, "vocab_size": 50257, "entropy": 0.758580207824707, "entropy_per_token": [0.9577832221984863, 0.7055426239967346, 0.0281821396201849, 0.41144680976867676, 0.9747774600982666, 0.12894627451896667, 1.0063729286193848, 0.39627620577812195, 0.6171188354492188, 2.649491310119629, 0.591342568397522, 0.46664097905158997, 1.2840341329574585, 5.486734153237194e-05, 0.00276753818616271, 0.6397385001182556, 0.7491754293441772, 0.4639410972595215, 3.0977959632873535, 0.00017619028221815825], "max_p": 0.7457332015037537, "max_p_per_token": [0.7284023761749268, 0.7482643723487854, 0.9966863989830017, 0.9102333784103394, 0.6209194660186768, 0.9776461720466614, 0.5738881230354309, 0.8647533059120178, 0.8394282460212708, 0.17560072243213654, 0.7486719489097595, 0.8280394077301025, 0.47141698002815247, 0.9999964237213135, 0.9997085928916931, 0.7906027436256409, 0.7233690023422241, 0.8261659145355225, 0.09088395535945892, 0.999985933303833], "n_positions_probed": 1, "per_restart_best": [5.640870571136475]}
+{"step": 254, "discrete_loss": 7.320648193359375, "best_sample_loss": 6.998744964599609, "soft_loss": 4.28981876373291, "best_discrete": 5.640870571136475, "best_soft": 4.28981876373291, "best_argmax": 7.262202739715576, "best_sampling": 5.640870571136475, "relax_gap": 0.4140110751908222, "n_match": 7, "g_first_norm": 155.6700897216797, "vocab_size": 50257, "entropy": 0.7679829001426697, "entropy_per_token": [0.9621974229812622, 0.7121260166168213, 0.027561699971556664, 0.42327678203582764, 0.9771318435668945, 0.1319499909877777, 1.0011329650878906, 0.41510123014450073, 0.6198244094848633, 2.724888324737549, 0.5895353555679321, 0.46413278579711914, 1.2792866230010986, 5.2316245273686945e-05, 0.0028475606814026833, 0.6534155607223511, 0.7528205513954163, 0.5160731077194214, 3.106123685836792, 0.000178902133484371], "max_p": 0.7419750094413757, "max_p_per_token": [0.7257607579231262, 0.7435054779052734, 0.9967709183692932, 0.9063506722450256, 0.6223086714744568, 0.9771319031715393, 0.5758102536201477, 0.8543639183044434, 0.8376027345657349, 0.16649870574474335, 0.7494633793830872, 0.8296347856521606, 0.46616876125335693, 0.999996542930603, 0.9997027516365051, 0.7854591012001038, 0.7215110659599304, 0.7900548577308655, 0.09142027050256729, 0.9999856948852539], "n_positions_probed": 1, "per_restart_best": [5.640870571136475]}
+{"step": 255, "discrete_loss": 7.320648193359375, "best_sample_loss": 6.288463592529297, "soft_loss": 4.232729911804199, "best_discrete": 5.640870571136475, "best_soft": 4.232729911804199, "best_argmax": 7.262202739715576, "best_sampling": 5.640870571136475, "relax_gap": 0.4218094081281291, "n_match": 7, "g_first_norm": 155.74801635742188, "vocab_size": 50257, "entropy": 0.7772904634475708, "entropy_per_token": [0.967557430267334, 0.7185471057891846, 0.027037188410758972, 0.4333459138870239, 0.9802325367927551, 0.13520994782447815, 0.9968875050544739, 0.43497371673583984, 0.6232711672782898, 2.797593593597412, 0.5879446864128113, 0.4606621265411377, 1.2745109796524048, 4.978026117896661e-05, 0.002840768313035369, 0.6669586896896362, 0.7564806938171387, 0.5676713585853577, 3.113852024078369, 0.0001818825548980385], "max_p": 0.7378208637237549, "max_p_per_token": [0.7227158546447754, 0.7387669086456299, 0.9968425035476685, 0.9029830694198608, 0.6226633787155151, 0.9765537977218628, 0.5766430497169495, 0.8428367972373962, 0.835527777671814, 0.15890243649482727, 0.750102162361145, 0.831821858882904, 0.460264652967453, 0.9999967813491821, 0.999703586101532, 0.780467689037323, 0.7195810079574585, 0.7472746968269348, 0.09278370440006256, 0.9999854564666748], "n_positions_probed": 1, "per_restart_best": [5.640870571136475]}
+{"step": 256, "discrete_loss": 7.320648193359375, "best_sample_loss": 6.746742248535156, "soft_loss": 4.1788482666015625, "best_discrete": 5.640870571136475, "best_soft": 4.1788482666015625, "best_argmax": 7.262202739715576, "best_sampling": 5.640870571136475, "relax_gap": 0.42916963686463816, "n_match": 7, "g_first_norm": 157.1106719970703, "vocab_size": 50257, "entropy": 0.7871439456939697, "entropy_per_token": [0.9752306938171387, 0.7244586944580078, 0.026496397331357002, 0.4422943592071533, 0.9834705591201782, 0.13879543542861938, 0.9920607805252075, 0.4550287127494812, 0.6273081302642822, 2.8553032875061035, 0.5864986777305603, 0.45646390318870544, 1.2691187858581543, 4.751286905957386e-05, 0.0028278622776269913, 0.6794307231903076, 0.7888555526733398, 0.6173912882804871, 3.121612548828125, 0.00018527415522839874], "max_p": 0.7332343459129333, "max_p_per_token": [0.7185280323028564, 0.7343633770942688, 0.9969156980514526, 0.8999084234237671, 0.6240267157554626, 0.9758955836296082, 0.5790570378303528, 0.8305646777153015, 0.8332765698432922, 0.15439048409461975, 0.7506749033927917, 0.834429144859314, 0.45404309034347534, 0.9999969005584717, 0.9997051358222961, 0.7760998010635376, 0.7137575745582581, 0.6952579617500305, 0.09381027519702911, 0.9999852180480957], "n_positions_probed": 1, "per_restart_best": [5.640870571136475]}
+{"step": 257, "discrete_loss": 6.5129714012146, "best_sample_loss": 5.0092597007751465, "soft_loss": 4.1214447021484375, "best_discrete": 5.0092597007751465, "best_soft": 4.1214447021484375, "best_argmax": 6.5129714012146, "best_sampling": 5.0092597007751465, "relax_gap": 0.36719441123604013, "n_match": 8, "g_first_norm": 159.4602508544922, "vocab_size": 50257, "entropy": 0.7622238993644714, "entropy_per_token": [0.9850133657455444, 0.7304142713546753, 0.02599293552339077, 0.4500601291656494, 0.9876554012298584, 0.14244906604290009, 0.9874581098556519, 0.47533369064331055, 0.6315335035324097, 2.907421350479126, 0.5850509405136108, 0.4517318904399872, 1.2634162902832031, 4.535656626103446e-05, 0.002807071665301919, 0.6906794309616089, 0.7918171882629395, 0.0056863888166844845, 3.1297223567962646, 0.00018932692182715982], "max_p": 0.7466391324996948, "max_p_per_token": [0.7133097052574158, 0.7299885749816895, 0.9969834685325623, 0.8971878886222839, 0.62510085105896, 0.9752109050750732, 0.5818991661071777, 0.8174130320549011, 0.8309901356697083, 0.15073685348033905, 0.7512909173965454, 0.8373311758041382, 0.4471617639064789, 0.9999970197677612, 0.9997077584266663, 0.772542417049408, 0.7118678092956543, 0.9993140697479248, 0.09476403892040253, 0.999984860420227], "n_positions_probed": 1, "per_restart_best": [5.0092597007751465]}
+{"step": 258, "discrete_loss": 6.5129714012146, "best_sample_loss": 5.143240451812744, "soft_loss": 3.567884922027588, "best_discrete": 5.0092597007751465, "best_soft": 3.567884922027588, "best_argmax": 6.5129714012146, "best_sampling": 5.0092597007751465, "relax_gap": 0.4521878414263856, "n_match": 8, "g_first_norm": 242.97824096679688, "vocab_size": 50257, "entropy": 0.6719513535499573, "entropy_per_token": [1.0260868072509766, 0.7217533588409424, 0.024684574455022812, 0.46262577176094055, 0.9815680980682373, 0.13859809935092926, 0.9855020642280579, 0.49541178345680237, 0.6313620209693909, 2.952204704284668, 0.5920417308807373, 0.43712079524993896, 1.2695114612579346, 4.6799123083474115e-05, 0.0028317293617874384, 0.6916617155075073, 0.7677081823348999, 0.00653661647811532, 1.2515733242034912, 0.00019718434487003833], "max_p": 0.774173378944397, "max_p_per_token": [0.6926777958869934, 0.7370806336402893, 0.9971559047698975, 0.8929125666618347, 0.6269057393074036, 0.9760946035385132, 0.5841513872146606, 0.8036006689071655, 0.830398678779602, 0.14692163467407227, 0.7449742555618286, 0.8456876277923584, 0.42944878339767456, 0.9999969005584717, 0.9997054934501648, 0.7779676914215088, 0.725682258605957, 0.9991958737373352, 0.6729244589805603, 0.9999842643737793], "n_positions_probed": 1, "per_restart_best": [5.0092597007751465]}
+{"step": 259, "discrete_loss": 6.5921478271484375, "best_sample_loss": 5.721730709075928, "soft_loss": 4.681802272796631, "best_discrete": 5.0092597007751465, "best_soft": 3.567884922027588, "best_argmax": 6.5129714012146, "best_sampling": 5.0092597007751465, "relax_gap": 0.2897910672579932, "n_match": 8, "g_first_norm": 157.7815704345703, "vocab_size": 50257, "entropy": 0.6844314932823181, "entropy_per_token": [1.042783260345459, 0.7225692272186279, 0.02494102716445923, 0.47647255659103394, 0.9718602895736694, 0.1394546926021576, 1.0074058771133423, 0.5175014734268188, 0.6135367155075073, 2.985408067703247, 0.595180332660675, 0.4488486051559448, 1.2689216136932373, 4.5110617065802217e-05, 0.0029270630329847336, 0.7126916646957397, 0.7664288282394409, 0.006289920303970575, 1.3851299285888672, 0.00023345070076175034], "max_p": 0.7682222723960876, "max_p_per_token": [0.6836170554161072, 0.7366872429847717, 0.9971233010292053, 0.8885628581047058, 0.656599760055542, 0.9759592413902283, 0.5593714118003845, 0.7873420715332031, 0.8360716700553894, 0.16804255545139313, 0.7418797612190247, 0.838939905166626, 0.40131261944770813, 0.9999970197677612, 0.9996945858001709, 0.7697471380233765, 0.7252863645553589, 0.9992303848266602, 0.5989989042282104, 0.9999810457229614], "n_positions_probed": 1, "per_restart_best": [5.0092597007751465]}
+{"step": 260, "discrete_loss": 6.354851722717285, "best_sample_loss": 5.0092597007751465, "soft_loss": 4.544757843017578, "best_discrete": 5.0092597007751465, "best_soft": 3.567884922027588, "best_argmax": 6.354851722717285, "best_sampling": 5.0092597007751465, "relax_gap": 0.28483652470269205, "n_match": 9, "g_first_norm": 157.08642578125, "vocab_size": 50257, "entropy": 0.6965152025222778, "entropy_per_token": [1.0222246646881104, 0.7236469984054565, 0.025045957416296005, 0.4893219470977783, 0.972892165184021, 0.14173342287540436, 1.0158138275146484, 0.5394532680511475, 0.6105295419692993, 3.0467023849487305, 0.5980755090713501, 0.44994255900382996, 1.2613568305969238, 4.3264928535791114e-05, 0.0030062985606491566, 0.7275581955909729, 0.7600055932998657, 0.0060632615350186825, 1.5366122722625732, 0.00027551339007914066], "max_p": 0.7585906386375427, "max_p_per_token": [0.6431301832199097, 0.7361266016960144, 0.9971100687980652, 0.8844238519668579, 0.667105495929718, 0.9755123257637024, 0.5497322678565979, 0.7698925733566284, 0.8359320163726807, 0.16404497623443604, 0.7388913631439209, 0.8383771181106567, 0.38198766112327576, 0.9999971389770508, 0.999685525894165, 0.7641383409500122, 0.728566586971283, 0.9992617964744568, 0.49791866540908813, 0.9999773502349854], "n_positions_probed": 1, "per_restart_best": [5.0092597007751465]}
+{"step": 261, "discrete_loss": 6.566491603851318, "best_sample_loss": 5.027184009552002, "soft_loss": 4.4413862228393555, "best_discrete": 5.0092597007751465, "best_soft": 3.567884922027588, "best_argmax": 6.354851722717285, "best_sampling": 5.0092597007751465, "relax_gap": 0.3236287365030004, "n_match": 9, "g_first_norm": 164.7380828857422, "vocab_size": 50257, "entropy": 0.6872483491897583, "entropy_per_token": [1.0476864576339722, 0.24621039628982544, 0.025163762271404266, 0.49681514501571655, 0.9808673858642578, 0.1453191637992859, 1.0158272981643677, 0.5619298815727234, 0.6328904628753662, 3.0769660472869873, 0.6008846759796143, 0.4409666955471039, 1.2644617557525635, 4.1833260183921084e-05, 0.0030497063416987658, 0.7398310899734497, 0.7496770024299622, 0.005839463789016008, 1.7102093696594238, 0.00032942439429461956], "max_p": 0.763558566570282, "max_p_per_token": [0.628293514251709, 0.9466435313224792, 0.9970948696136475, 0.8820972442626953, 0.6794435381889343, 0.974770188331604, 0.5522112250328064, 0.7504141330718994, 0.825951099395752, 0.1707988977432251, 0.7359042167663574, 0.8437421917915344, 0.3951650559902191, 0.9999972581863403, 0.9996806383132935, 0.7590120434761047, 0.7341568470001221, 0.9992928504943848, 0.3965297043323517, 0.9999724626541138], "n_positions_probed": 1, "per_restart_best": [5.0092597007751465]}
+{"step": 262, "discrete_loss": 6.149240016937256, "best_sample_loss": 5.014464855194092, "soft_loss": 4.449188232421875, "best_discrete": 5.0092597007751465, "best_soft": 3.567884922027588, "best_argmax": 6.149240016937256, "best_sampling": 5.0092597007751465, "relax_gap": 0.276465348536212, "n_match": 9, "g_first_norm": 219.4399871826172, "vocab_size": 50257, "entropy": 0.6994558572769165, "entropy_per_token": [1.0959010124206543, 0.25713345408439636, 0.02558727189898491, 0.5189960598945618, 0.9664521217346191, 0.14854151010513306, 1.0276038646697998, 0.5689801573753357, 0.6436172723770142, 3.1537036895751953, 0.606321394443512, 0.37410643696784973, 1.2098109722137451, 3.862437733914703e-05, 0.0032042786478996277, 0.7249019145965576, 0.7412534952163696, 0.005487953312695026, 1.9170594215393066, 0.0004167212755419314], "max_p": 0.7644630074501038, "max_p_per_token": [0.6098637580871582, 0.9434139132499695, 0.9970396161079407, 0.8748923540115356, 0.6887152791023254, 0.9740890264511108, 0.5376722812652588, 0.7439085841178894, 0.8202546238899231, 0.15573148429393768, 0.7299225330352783, 0.879298210144043, 0.46629780530929565, 0.9999974966049194, 0.9996625185012817, 0.7675957083702087, 0.7390351891517639, 0.9993409514427185, 0.36256396770477295, 0.999964714050293], "n_positions_probed": 1, "per_restart_best": [5.0092597007751465]}
+{"step": 263, "discrete_loss": 6.355998516082764, "best_sample_loss": 4.967931270599365, "soft_loss": 4.170235633850098, "best_discrete": 4.967931270599365, "best_soft": 3.567884922027588, "best_argmax": 6.149240016937256, "best_sampling": 4.967931270599365, "relax_gap": 0.34388977226819173, "n_match": 10, "g_first_norm": 215.88455200195312, "vocab_size": 50257, "entropy": 0.7141112685203552, "entropy_per_token": [1.1087151765823364, 0.2626037299633026, 0.025516629219055176, 0.32165277004241943, 0.962428629398346, 0.15140879154205322, 1.028005838394165, 0.5744220018386841, 0.6412267684936523, 3.1856448650360107, 0.6080372333526611, 0.35949796438217163, 1.1923043727874756, 3.638081034296192e-05, 0.003245476633310318, 0.7077677249908447, 0.7325336933135986, 0.005300058517605066, 2.4114155769348145, 0.0004619465034920722], "max_p": 0.7628813982009888, "max_p_per_token": [0.6114969253540039, 0.941742479801178, 0.9970487952232361, 0.9115204811096191, 0.6892175674438477, 0.9734765291213989, 0.5375993847846985, 0.7387405037879944, 0.8199124336242676, 0.14026017487049103, 0.7274425029754639, 0.8863935470581055, 0.49221503734588623, 0.999997615814209, 0.9996578693389893, 0.7775006294250488, 0.7448463439941406, 0.9993665814399719, 0.2692306339740753, 0.9999606609344482], "n_positions_probed": 1, "per_restart_best": [4.967931270599365]}
+{"step": 264, "discrete_loss": 6.523321628570557, "best_sample_loss": 5.4278130531311035, "soft_loss": 3.885906219482422, "best_discrete": 4.967931270599365, "best_soft": 3.567884922027588, "best_argmax": 6.149240016937256, "best_sampling": 4.967931270599365, "relax_gap": 0.4043055914239916, "n_match": 10, "g_first_norm": 172.911865234375, "vocab_size": 50257, "entropy": 0.7328585982322693, "entropy_per_token": [1.1301997900009155, 0.2738000750541687, 0.025655832141637802, 0.3343551456928253, 0.9618913531303406, 0.1532393842935562, 1.0245472192764282, 0.5788555145263672, 0.6565455198287964, 3.2070794105529785, 0.6071637272834778, 0.36615508794784546, 1.1930642127990723, 3.4525295632192865e-05, 0.003173490520566702, 0.6984691619873047, 0.7182092666625977, 0.005168079398572445, 2.719073534011841, 0.0004915124736726284], "max_p": 0.7591090798377991, "max_p_per_token": [0.5978220701217651, 0.9383770823478699, 0.9970306158065796, 0.9062771201133728, 0.6939705014228821, 0.9730620384216309, 0.5521858334541321, 0.7344288229942322, 0.8124833703041077, 0.13459253311157227, 0.7279113531112671, 0.883240818977356, 0.4941858947277069, 0.9999977350234985, 0.9996670484542847, 0.783392071723938, 0.7540900111198425, 0.9993844032287598, 0.20012366771697998, 0.9999581575393677], "n_positions_probed": 1, "per_restart_best": [4.967931270599365]}
+{"step": 265, "discrete_loss": 6.523321628570557, "best_sample_loss": 5.133906841278076, "soft_loss": 3.6060805320739746, "best_discrete": 4.967931270599365, "best_soft": 3.567884922027588, "best_argmax": 6.149240016937256, "best_sampling": 4.967931270599365, "relax_gap": 0.44720178807676414, "n_match": 10, "g_first_norm": 161.8363800048828, "vocab_size": 50257, "entropy": 0.7460320591926575, "entropy_per_token": [1.1349704265594482, 0.2915557026863098, 0.02596927061676979, 0.3499748110771179, 0.9595147371292114, 0.15511034429073334, 1.0256861448287964, 0.5860507488250732, 0.6653271317481995, 3.2235772609710693, 0.6084477305412292, 0.38084840774536133, 1.1968774795532227, 3.361113340361044e-05, 0.0031375286635011435, 0.694913387298584, 0.7007193565368652, 0.005010940134525299, 2.912396192550659, 0.0005196393467485905], "max_p": 0.7564508318901062, "max_p_per_token": [0.5893404483795166, 0.9329236745834351, 0.9969896078109741, 0.899652361869812, 0.6964523792266846, 0.9726393818855286, 0.554356038570404, 0.7272253036499023, 0.8077394366264343, 0.12816615402698517, 0.7264679074287415, 0.8759911060333252, 0.49353548884391785, 0.9999978542327881, 0.9996716976165771, 0.7873589396476746, 0.7656993269920349, 0.9994056224822998, 0.17544861137866974, 0.999955415725708], "n_positions_probed": 1, "per_restart_best": [4.967931270599365]}
+{"step": 266, "discrete_loss": 6.523321628570557, "best_sample_loss": 5.090285778045654, "soft_loss": 3.4602348804473877, "best_discrete": 4.967931270599365, "best_soft": 3.4602348804473877, "best_argmax": 6.149240016937256, "best_sampling": 4.967931270599365, "relax_gap": 0.46955936293369266, "n_match": 10, "g_first_norm": 155.70318603515625, "vocab_size": 50257, "entropy": 0.7550821900367737, "entropy_per_token": [1.1394503116607666, 0.3111327886581421, 0.02618619054555893, 0.3646951913833618, 0.9526463747024536, 0.1556588113307953, 1.078841209411621, 0.5945783853530884, 0.6736917495727539, 3.2304494380950928, 0.6132145524024963, 0.39416223764419556, 1.2004846334457397, 3.329444007249549e-05, 0.0031068173702806234, 0.6975405216217041, 0.6864625811576843, 0.004971316084265709, 2.9737915992736816, 0.0005461599212139845], "max_p": 0.7545372843742371, "max_p_per_token": [0.5805012583732605, 0.9266988635063171, 0.9969616532325745, 0.8931867480278015, 0.6996802687644958, 0.9724991917610168, 0.5488497614860535, 0.71832674741745, 0.8029769659042358, 0.12538385391235352, 0.7216179370880127, 0.8691789507865906, 0.4930039346218109, 0.9999978542327881, 0.999675989151001, 0.7884896397590637, 0.7753095030784607, 0.9994109869003296, 0.1790420114994049, 0.9999531507492065], "n_positions_probed": 1, "per_restart_best": [4.967931270599365]}
+{"step": 267, "discrete_loss": 6.523321628570557, "best_sample_loss": 5.083743572235107, "soft_loss": 3.3989129066467285, "best_discrete": 4.967931270599365, "best_soft": 3.3989129066467285, "best_argmax": 6.149240016937256, "best_sampling": 4.967931270599365, "relax_gap": 0.47895978457350324, "n_match": 10, "g_first_norm": 155.63206481933594, "vocab_size": 50257, "entropy": 0.7603259086608887, "entropy_per_token": [1.1426094770431519, 0.3323298990726471, 0.026297058910131454, 0.3779512643814087, 0.9437226057052612, 0.15638616681098938, 1.0764551162719727, 0.6055041551589966, 0.6830580234527588, 3.2323899269104004, 0.6189525723457336, 0.40551671385765076, 1.2047563791275024, 3.3157197321997955e-05, 0.0030683421064168215, 0.7012654542922974, 0.6744889616966248, 0.004976046737283468, 3.0161852836608887, 0.0005721955676563084], "max_p": 0.7525199055671692, "max_p_per_token": [0.5728036165237427, 0.9197498559951782, 0.996947705745697, 0.8871577382087708, 0.7036899328231812, 0.9723154902458191, 0.5521663427352905, 0.7087752819061279, 0.7977070212364197, 0.12398677319288254, 0.7156204581260681, 0.863161563873291, 0.4913892447948456, 0.9999978542327881, 0.9996811151504517, 0.7890781760215759, 0.7834168076515198, 0.9994102716445923, 0.17339295148849487, 0.999950647354126], "n_positions_probed": 1, "per_restart_best": [4.967931270599365]}
+{"step": 268, "discrete_loss": 6.523321628570557, "best_sample_loss": 4.966040134429932, "soft_loss": 3.3523945808410645, "best_discrete": 4.966040134429932, "best_soft": 3.3523945808410645, "best_argmax": 6.149240016937256, "best_sampling": 4.966040134429932, "relax_gap": 0.4860908641759446, "n_match": 10, "g_first_norm": 155.06643676757812, "vocab_size": 50257, "entropy": 0.7728909850120544, "entropy_per_token": [1.1435272693634033, 0.3568503260612488, 0.02636897563934326, 0.39038971066474915, 0.935067892074585, 0.15719401836395264, 1.0742366313934326, 0.6138246059417725, 0.8518145680427551, 3.234189748764038, 0.624550461769104, 0.4152581989765167, 1.209122657775879, 3.3073301892727613e-05, 0.003025998827069998, 0.7059967517852783, 0.6639743447303772, 0.0049846721813082695, 3.0468132495880127, 0.0005973693914711475], "max_p": 0.7361119985580444, "max_p_per_token": [0.5665706396102905, 0.9114118218421936, 0.9969388246536255, 0.881324052810669, 0.7073842883110046, 0.9721088409423828, 0.5547917485237122, 0.6991789937019348, 0.5052317976951599, 0.12227614223957062, 0.7095677256584167, 0.8578422665596008, 0.4898107051849365, 0.9999978542327881, 0.9996868371963501, 0.7891380190849304, 0.7905614376068115, 0.9994090795516968, 0.16905958950519562, 0.9999483823776245], "n_positions_probed": 1, "per_restart_best": [4.966040134429932]}
+{"step": 269, "discrete_loss": 6.368860721588135, "best_sample_loss": 5.348876476287842, "soft_loss": 3.87199330329895, "best_discrete": 4.966040134429932, "best_soft": 3.3523945808410645, "best_argmax": 6.149240016937256, "best_sampling": 4.966040134429932, "relax_gap": 0.392043024245405, "n_match": 10, "g_first_norm": 246.596435546875, "vocab_size": 50257, "entropy": 0.6499435305595398, "entropy_per_token": [1.1847765445709229, 0.36905163526535034, 0.027509009465575218, 0.41232186555862427, 0.9166635870933533, 0.16002070903778076, 1.0854847431182861, 0.6326961517333984, 0.5168468952178955, 0.9430365562438965, 0.6396464109420776, 0.4305686056613922, 1.221764326095581, 3.281715544289909e-05, 0.002996172057464719, 0.7057336568832397, 0.6561583280563354, 0.004881190601736307, 3.088057518005371, 0.0006228312849998474], "max_p": 0.7832774519920349, "max_p_per_token": [0.5446213483810425, 0.9073525071144104, 0.9967886209487915, 0.8710973262786865, 0.7163199782371521, 0.9714756608009338, 0.5384730696678162, 0.6751976013183594, 0.8588149547576904, 0.8154444694519043, 0.692858099937439, 0.8494350910186768, 0.4783664345741272, 0.9999978542327881, 0.9996911287307739, 0.7923035621643066, 0.7966432571411133, 0.9994230270385742, 0.16129820048809052, 0.999946117401123], "n_positions_probed": 1, "per_restart_best": [4.966040134429932]}
+{"step": 270, "discrete_loss": 6.30857515335083, "best_sample_loss": 5.09357213973999, "soft_loss": 4.548182487487793, "best_discrete": 4.966040134429932, "best_soft": 3.3523945808410645, "best_argmax": 6.149240016937256, "best_sampling": 4.966040134429932, "relax_gap": 0.2790475857179883, "n_match": 10, "g_first_norm": 229.9084930419922, "vocab_size": 50257, "entropy": 0.6590280532836914, "entropy_per_token": [1.2121608257293701, 0.3699098825454712, 0.028164945542812347, 0.4168849587440491, 0.8056871891021729, 0.15783575177192688, 1.1107909679412842, 0.6423095464706421, 0.5371626019477844, 1.057960033416748, 0.7340262532234192, 0.5109061598777771, 1.2462904453277588, 3.591007043723948e-05, 0.0030615157447755337, 0.725292444229126, 0.6767827272415161, 0.005166036542505026, 2.939486026763916, 0.0006471339147537947], "max_p": 0.7766194343566895, "max_p_per_token": [0.5244261026382446, 0.9070394039154053, 0.996701180934906, 0.868923544883728, 0.7568619251251221, 0.9719664454460144, 0.5056511163711548, 0.6614095568656921, 0.8504928350448608, 0.7817474007606506, 0.6481801867485046, 0.7989237308502197, 0.46747541427612305, 0.999997615814209, 0.9996851682662964, 0.7868305444717407, 0.7875500917434692, 0.9993845224380493, 0.21919938921928406, 0.999943733215332], "n_positions_probed": 1, "per_restart_best": [4.966040134429932]}
+{"step": 271, "discrete_loss": 6.356184959411621, "best_sample_loss": 5.203862190246582, "soft_loss": 4.43778657913208, "best_discrete": 4.966040134429932, "best_soft": 3.3523945808410645, "best_argmax": 6.149240016937256, "best_sampling": 4.966040134429932, "relax_gap": 0.3018160095292638, "n_match": 9, "g_first_norm": 153.3793487548828, "vocab_size": 50257, "entropy": 0.673371434211731, "entropy_per_token": [1.2138140201568604, 0.3808937072753906, 0.029554419219493866, 0.43050894141197205, 0.7185368537902832, 0.16195116937160492, 1.1397396326065063, 0.6540422439575195, 0.5630741119384766, 1.22154700756073, 0.7401718497276306, 0.4203190803527832, 1.2515838146209717, 3.6454035580391064e-05, 0.003111654194071889, 0.7287619113922119, 0.6684499382972717, 0.0049926843494176865, 3.135653018951416, 0.0006856987602077425], "max_p": 0.7699186205863953, "max_p_per_token": [0.513335108757019, 0.9034448266029358, 0.9965152740478516, 0.8622141480445862, 0.7930658459663391, 0.9710595607757568, 0.46515387296676636, 0.6427206993103027, 0.8399900794029236, 0.7299525737762451, 0.6375479102134705, 0.8527605533599854, 0.47659313678741455, 0.999997615814209, 0.9996801614761353, 0.7881731986999512, 0.7947530150413513, 0.9994078874588013, 0.13206620514392853, 0.999940037727356], "n_positions_probed": 1, "per_restart_best": [4.966040134429932]}
+{"step": 272, "discrete_loss": 6.5963311195373535, "best_sample_loss": 4.978102207183838, "soft_loss": 4.4216485023498535, "best_discrete": 4.966040134429932, "best_soft": 3.3523945808410645, "best_argmax": 6.149240016937256, "best_sampling": 4.966040134429932, "relax_gap": 0.32968063273027837, "n_match": 9, "g_first_norm": 230.658935546875, "vocab_size": 50257, "entropy": 0.6887445449829102, "entropy_per_token": [1.28336501121521, 0.355144739151001, 0.02915157377719879, 0.43616604804992676, 0.6215717792510986, 0.16156882047653198, 1.179420828819275, 0.6601799130439758, 0.5716984272003174, 1.5650484561920166, 0.7518780827522278, 0.46518298983573914, 1.290695071220398, 3.7591529689962044e-05, 0.0032222801819443703, 0.7342323064804077, 0.6703841686248779, 0.005256946198642254, 2.98996639251709, 0.0007200100226327777], "max_p": 0.760745644569397, "max_p_per_token": [0.46124958992004395, 0.9127407670021057, 0.9965692758560181, 0.8590036630630493, 0.829120934009552, 0.9711111783981323, 0.428265780210495, 0.6317130923271179, 0.8364070057868958, 0.6071378588676453, 0.6175459027290344, 0.8259301781654358, 0.44982197880744934, 0.9999974966049194, 0.9996684789657593, 0.7883625626564026, 0.7968743443489075, 0.9993721842765808, 0.20408424735069275, 0.9999369382858276], "n_positions_probed": 1, "per_restart_best": [4.966040134429932]}
+{"step": 273, "discrete_loss": 6.523097515106201, "best_sample_loss": 5.9392876625061035, "soft_loss": 4.208558082580566, "best_discrete": 4.966040134429932, "best_soft": 3.3523945808410645, "best_argmax": 6.149240016937256, "best_sampling": 4.966040134429932, "relax_gap": 0.35482214196026046, "n_match": 9, "g_first_norm": 229.0778350830078, "vocab_size": 50257, "entropy": 0.7198027968406677, "entropy_per_token": [1.302868366241455, 0.3567441701889038, 0.029953792691230774, 0.45219600200653076, 0.5837531685829163, 0.16921743750572205, 1.2149100303649902, 0.6681137084960938, 0.5756002068519592, 1.932190179824829, 0.7568126916885376, 0.4856020510196686, 1.3058795928955078, 3.661546725197695e-05, 0.003212772309780121, 0.7353479266166687, 0.6487360000610352, 0.005159006919711828, 3.168956995010376, 0.0007658767281100154], "max_p": 0.7481175065040588, "max_p_per_token": [0.42685434222221375, 0.9126465320587158, 0.9964619278907776, 0.8505160212516785, 0.8452439308166504, 0.9693622589111328, 0.42315441370010376, 0.6160104274749756, 0.8341730237007141, 0.5079196691513062, 0.6044018268585205, 0.8126091957092285, 0.4481303095817566, 0.999997615814209, 0.999670147895813, 0.7897867560386658, 0.8107655048370361, 0.9993854761123657, 0.11532731354236603, 0.9999325275421143], "n_positions_probed": 1, "per_restart_best": [4.966040134429932]}
+{"step": 274, "discrete_loss": 6.5963311195373535, "best_sample_loss": 5.838447093963623, "soft_loss": 3.7400619983673096, "best_discrete": 4.966040134429932, "best_soft": 3.3523945808410645, "best_argmax": 6.149240016937256, "best_sampling": 4.966040134429932, "relax_gap": 0.4330087543225656, "n_match": 9, "g_first_norm": 224.5405731201172, "vocab_size": 50257, "entropy": 0.7570778131484985, "entropy_per_token": [1.3460021018981934, 0.35801780223846436, 0.030579719692468643, 0.4632551670074463, 0.614315927028656, 0.17709128558635712, 1.2343831062316895, 0.671777606010437, 0.5864686965942383, 2.58595871925354, 0.7438346743583679, 0.512222945690155, 1.3260769844055176, 3.5472050512908027e-05, 0.003271156456321478, 0.7464103698730469, 0.6420023441314697, 0.005315754096955061, 3.093716621398926, 0.0008198642171919346], "max_p": 0.7337822318077087, "max_p_per_token": [0.40657317638397217, 0.912456750869751, 0.9963783621788025, 0.8442568778991699, 0.8345354795455933, 0.9674784541130066, 0.409280925989151, 0.607883632183075, 0.8288592100143433, 0.255414217710495, 0.6158823370933533, 0.7939870953559875, 0.4391801655292511, 0.999997615814209, 0.999669075012207, 0.7867130637168884, 0.8166938424110413, 0.9993643164634705, 0.16111312806606293, 0.9999271631240845], "n_positions_probed": 1, "per_restart_best": [4.966040134429932]}
+{"step": 275, "discrete_loss": 5.331405162811279, "best_sample_loss": 5.630919933319092, "soft_loss": 3.3964240550994873, "best_discrete": 4.966040134429932, "best_soft": 3.3523945808410645, "best_argmax": 5.331405162811279, "best_sampling": 4.966040134429932, "relax_gap": 0.3629401721724458, "n_match": 9, "g_first_norm": 179.75778198242188, "vocab_size": 50257, "entropy": 0.7699525952339172, "entropy_per_token": [1.3552939891815186, 0.3839748501777649, 0.030499882996082306, 0.472181499004364, 0.6265783905982971, 0.18919795751571655, 1.2645940780639648, 0.6656493544578552, 0.5843385457992554, 2.714435338973999, 0.737666666507721, 0.5189839005470276, 1.3195527791976929, 3.350044789840467e-05, 0.0031719813123345375, 0.7476252317428589, 0.620927631855011, 0.005227555986493826, 3.1582741737365723, 0.0008450163877569139], "max_p": 0.7293353080749512, "max_p_per_token": [0.4073406457901001, 0.9036481976509094, 0.9963893890380859, 0.8389962315559387, 0.8323317170143127, 0.964561939239502, 0.3670376241207123, 0.6209766268730164, 0.8295140862464905, 0.2515660226345062, 0.620549201965332, 0.789044976234436, 0.45227688550949097, 0.9999977350234985, 0.999680757522583, 0.7876848578453064, 0.8285436034202576, 0.9993762373924255, 0.09726432710886002, 0.9999250173568726], "n_positions_probed": 1, "per_restart_best": [4.966040134429932]}
+{"step": 276, "discrete_loss": 5.331405162811279, "best_sample_loss": 5.723474502563477, "soft_loss": 3.1826977729797363, "best_discrete": 4.966040134429932, "best_soft": 3.1826977729797363, "best_argmax": 5.331405162811279, "best_sampling": 4.966040134429932, "relax_gap": 0.4030283432254692, "n_match": 9, "g_first_norm": 164.486328125, "vocab_size": 50257, "entropy": 0.7754987478256226, "entropy_per_token": [1.376088261604309, 0.39486193656921387, 0.03061557002365589, 0.48062291741371155, 0.6563980579376221, 0.19775424897670746, 1.263450264930725, 0.6634323596954346, 0.5767679214477539, 2.849137783050537, 0.7219059467315674, 0.5335900783538818, 1.3199315071105957, 3.174137236783281e-05, 0.003068190300837159, 0.75309157371521, 0.5887526273727417, 0.005243329331278801, 3.0943663120269775, 0.000863457506056875], "max_p": 0.7295231223106384, "max_p_per_token": [0.40923401713371277, 0.8999029994010925, 0.9963743090629578, 0.8338907957077026, 0.8219246864318848, 0.9623827338218689, 0.3742692172527313, 0.6253244280815125, 0.8321453332901001, 0.213288813829422, 0.6368711590766907, 0.7778807282447815, 0.45508840680122375, 0.9999978542327881, 0.9996932744979858, 0.7866519093513489, 0.8406847715377808, 0.9993740916252136, 0.12556025385856628, 0.9999233484268188], "n_positions_probed": 1, "per_restart_best": [4.966040134429932]}
+{"step": 277, "discrete_loss": 5.331405162811279, "best_sample_loss": 5.868719577789307, "soft_loss": 3.0648741722106934, "best_discrete": 4.966040134429932, "best_soft": 3.0648741722106934, "best_argmax": 5.331405162811279, "best_sampling": 4.966040134429932, "relax_gap": 0.4251282581955245, "n_match": 9, "g_first_norm": 144.6339569091797, "vocab_size": 50257, "entropy": 0.7824656367301941, "entropy_per_token": [1.3818387985229492, 0.4236243963241577, 0.030791403725743294, 0.49383705854415894, 0.6792154908180237, 0.2070927917957306, 1.2543983459472656, 0.6611173152923584, 0.5715993046760559, 2.9201807975769043, 0.7145881652832031, 0.5385231375694275, 1.3144099712371826, 3.0074450478423387e-05, 0.002958504715934396, 0.7605459690093994, 0.5723211765289307, 0.005177950020879507, 3.11618709564209, 0.0008760589407756925], "max_p": 0.7282688617706299, "max_p_per_token": [0.4097709655761719, 0.8895910382270813, 0.996350884437561, 0.8259351849555969, 0.8147584795951843, 0.9599578380584717, 0.3898977041244507, 0.6297454237937927, 0.8332775235176086, 0.18023480474948883, 0.6441627144813538, 0.77401202917099, 0.4627213180065155, 0.9999979734420776, 0.999705970287323, 0.7846606969833374, 0.8484190106391907, 0.9993828535079956, 0.12287310510873795, 0.9999222755432129], "n_positions_probed": 1, "per_restart_best": [4.966040134429932]}
+{"step": 278, "discrete_loss": 5.331405162811279, "best_sample_loss": 5.107804298400879, "soft_loss": 2.9830832481384277, "best_discrete": 4.966040134429932, "best_soft": 2.9830832481384277, "best_argmax": 5.331405162811279, "best_sampling": 4.966040134429932, "relax_gap": 0.4404696028456724, "n_match": 9, "g_first_norm": 144.5305633544922, "vocab_size": 50257, "entropy": 0.7882477641105652, "entropy_per_token": [1.395676851272583, 0.44031694531440735, 0.030746757984161377, 0.505668580532074, 0.7008861303329468, 0.21611785888671875, 1.247098684310913, 0.6603794693946838, 0.5639952421188354, 2.9495768547058105, 0.7056662440299988, 0.5457939505577087, 1.3143105506896973, 2.8605292754946277e-05, 0.00285466224886477, 0.7719494104385376, 0.5586700439453125, 0.0051367757841944695, 3.149197816848755, 0.0008838131325319409], "max_p": 0.7275983691215515, "max_p_per_token": [0.40537354350090027, 0.8834319710731506, 0.9963571429252625, 0.8185663819313049, 0.8072417974472046, 0.9575381875038147, 0.39643630385398865, 0.6311091780662537, 0.8354681134223938, 0.16648000478744507, 0.6531294584274292, 0.7681503891944885, 0.4642444849014282, 0.9999980926513672, 0.9997180104255676, 0.7811031937599182, 0.8546361923217773, 0.9993884563446045, 0.13367392122745514, 0.9999219179153442], "n_positions_probed": 1, "per_restart_best": [4.966040134429932]}
+{"step": 279, "discrete_loss": 5.331405162811279, "best_sample_loss": 5.5080156326293945, "soft_loss": 2.943563938140869, "best_discrete": 4.966040134429932, "best_soft": 2.943563938140869, "best_argmax": 5.331405162811279, "best_sampling": 4.966040134429932, "relax_gap": 0.4478821533442205, "n_match": 9, "g_first_norm": 140.15538024902344, "vocab_size": 50257, "entropy": 0.7904503345489502, "entropy_per_token": [1.4093568325042725, 0.45855554938316345, 0.030549874529242516, 0.5182617902755737, 0.7174053192138672, 0.22614705562591553, 1.2374931573867798, 0.6598023772239685, 0.5561099052429199, 2.9624195098876953, 0.6991515159606934, 0.5503792762756348, 1.3148527145385742, 2.7256763132754713e-05, 0.0027544163167476654, 0.7841121554374695, 0.5463902950286865, 0.005055722780525684, 3.128722667694092, 0.0014598442940041423], "max_p": 0.7269112467765808, "max_p_per_token": [0.39832475781440735, 0.8765689730644226, 0.9963839054107666, 0.8105063438415527, 0.801670253276825, 0.9547937512397766, 0.4050002694129944, 0.6321738362312317, 0.837750256061554, 0.15788565576076508, 0.6593953967094421, 0.7643842697143555, 0.4655017554759979, 0.9999982118606567, 0.999729573726654, 0.7771774530410767, 0.8600736856460571, 0.9993994235992432, 0.14163662493228912, 0.9998694658279419], "n_positions_probed": 1, "per_restart_best": [4.966040134429932]}
+{"step": 280, "discrete_loss": 5.331405162811279, "best_sample_loss": 4.953085422515869, "soft_loss": 2.9156270027160645, "best_discrete": 4.953085422515869, "best_soft": 2.9156270027160645, "best_argmax": 5.331405162811279, "best_sampling": 4.953085422515869, "relax_gap": 0.45312222318916046, "n_match": 8, "g_first_norm": 140.16871643066406, "vocab_size": 50257, "entropy": 0.7927412390708923, "entropy_per_token": [1.4292607307434082, 0.47210270166397095, 0.030383888632059097, 0.5303075313568115, 0.7337995767593384, 0.23642493784427643, 1.2307143211364746, 0.659700870513916, 0.5474383234977722, 2.9732890129089355, 0.6921553611755371, 0.5545780062675476, 1.3177380561828613, 2.5864623239613138e-05, 0.002655967604368925, 0.7971593141555786, 0.5355491042137146, 0.004967971239238977, 3.1050829887390137, 0.00148987234570086], "max_p": 0.7259299159049988, "max_p_per_token": [0.3899906575679779, 0.8713842630386353, 0.9964063763618469, 0.8025733828544617, 0.7957205176353455, 0.9519028663635254, 0.4071730673313141, 0.6323545575141907, 0.8403920531272888, 0.1528034806251526, 0.6661532521247864, 0.7608816027641296, 0.46494683623313904, 0.9999984502792358, 0.9997406601905823, 0.7729427814483643, 0.8647922873497009, 0.9994112253189087, 0.14916381239891052, 0.9998668432235718], "n_positions_probed": 1, "per_restart_best": [4.953085422515869]}
+{"step": 281, "discrete_loss": 5.331405162811279, "best_sample_loss": 5.002220153808594, "soft_loss": 2.895927667617798, "best_discrete": 4.953085422515869, "best_soft": 2.895927667617798, "best_argmax": 5.331405162811279, "best_sampling": 4.953085422515869, "relax_gap": 0.45681718436668967, "n_match": 8, "g_first_norm": 138.9313507080078, "vocab_size": 50257, "entropy": 0.7948135733604431, "entropy_per_token": [1.4429512023925781, 0.47915810346603394, 0.030200602486729622, 0.5422535538673401, 0.7493985891342163, 0.2471400648355484, 1.2243915796279907, 0.6595604419708252, 0.5388256907463074, 2.984708309173584, 0.6857461333274841, 0.5577383637428284, 1.3211019039154053, 2.479356771800667e-05, 0.0025604304391890764, 0.8106956481933594, 0.5261133313179016, 0.004872877616435289, 3.087308883666992, 0.001520233927294612], "max_p": 0.7248117327690125, "max_p_per_token": [0.3825838565826416, 0.867222249507904, 0.9964313507080078, 0.794460117816925, 0.7899146676063538, 0.9488109946250916, 0.4083704948425293, 0.6326087117195129, 0.8430331349372864, 0.148829385638237, 0.6722256541252136, 0.7582176923751831, 0.4643406867980957, 0.9999984502792358, 0.9997515082359314, 0.7685381174087524, 0.8688555955886841, 0.9994239807128906, 0.1527530699968338, 0.9998642206192017], "n_positions_probed": 1, "per_restart_best": [4.953085422515869]}
+{"step": 282, "discrete_loss": 5.331405162811279, "best_sample_loss": 4.9656572341918945, "soft_loss": 2.879695415496826, "best_discrete": 4.953085422515869, "best_soft": 2.879695415496826, "best_argmax": 5.331405162811279, "best_sampling": 4.953085422515869, "relax_gap": 0.45986183237697376, "n_match": 8, "g_first_norm": 139.04248046875, "vocab_size": 50257, "entropy": 0.7967004179954529, "entropy_per_token": [1.4557175636291504, 0.4912797510623932, 0.01730448193848133, 0.5538592338562012, 0.7645190358161926, 0.2583211660385132, 1.2189024686813354, 0.6596046686172485, 0.5303910970687866, 2.9989912509918213, 0.6798242926597595, 0.5600795149803162, 1.3249107599258423, 2.3679061996517703e-05, 0.002468518214300275, 0.8247628211975098, 0.5178548693656921, 0.004773310385644436, 3.06886887550354, 0.0015513089019805193], "max_p": 0.7236750721931458, "max_p_per_token": [0.37589457631111145, 0.8623256087303162, 0.9978283047676086, 0.7863235473632812, 0.7840941548347473, 0.9454981684684753, 0.40799421072006226, 0.6325224041938782, 0.8456262946128845, 0.14477600157260895, 0.6777158379554749, 0.7562334537506104, 0.46377497911453247, 0.9999985694885254, 0.9997617602348328, 0.7639399766921997, 0.872380256652832, 0.9994372725486755, 0.1575145721435547, 0.999861478805542], "n_positions_probed": 1, "per_restart_best": [4.953085422515869]}
+{"step": 283, "discrete_loss": 5.331405162811279, "best_sample_loss": 4.962096691131592, "soft_loss": 2.864628314971924, "best_discrete": 4.953085422515869, "best_soft": 2.864628314971924, "best_argmax": 5.331405162811279, "best_sampling": 4.953085422515869, "relax_gap": 0.46268793545201325, "n_match": 8, "g_first_norm": 138.52159118652344, "vocab_size": 50257, "entropy": 0.8070077896118164, "entropy_per_token": [1.4678317308425903, 0.5026934146881104, 0.017442332580685616, 0.7131912708282471, 0.7793402671813965, 0.269866943359375, 1.2137439250946045, 0.659562349319458, 0.5222917199134827, 3.0149364471435547, 0.6743284463882446, 0.561870813369751, 1.3290623426437378, 2.2749387426301837e-05, 0.0023795526940375566, 0.839108943939209, 0.5106879472732544, 0.004669127054512501, 3.055542469024658, 0.0015831406926736236], "max_p": 0.7121722102165222, "max_p_per_token": [0.3697558045387268, 0.8576288223266602, 0.9978042244911194, 0.5752348899841309, 0.778264045715332, 0.9419827461242676, 0.40713706612586975, 0.6325953602790833, 0.8481060862541199, 0.14108359813690186, 0.6827207207679749, 0.7547115087509155, 0.4632624387741089, 0.9999985694885254, 0.9997716546058655, 0.7592397332191467, 0.8754197359085083, 0.999451220035553, 0.15941785275936127, 0.9998587369918823], "n_positions_probed": 1, "per_restart_best": [4.953085422515869]}
+{"step": 284, "discrete_loss": 5.331405162811279, "best_sample_loss": 5.20694637298584, "soft_loss": 2.837670087814331, "best_discrete": 4.953085422515869, "best_soft": 2.837670087814331, "best_argmax": 5.331405162811279, "best_sampling": 4.953085422515869, "relax_gap": 0.4677444311289199, "n_match": 8, "g_first_norm": 140.10194396972656, "vocab_size": 50257, "entropy": 0.8129774332046509, "entropy_per_token": [1.477416753768921, 0.511104941368103, 0.017585325986146927, 0.7146919965744019, 0.8702259063720703, 0.2817869782447815, 1.2108466625213623, 0.6603387594223022, 0.5147527456283569, 3.034358024597168, 0.6689853072166443, 0.5616486072540283, 1.3332264423370361, 2.1857144020032138e-05, 0.0023053884506225586, 0.8525142073631287, 0.5044320821762085, 0.004542899318039417, 3.0371580123901367, 0.0016044563381001353], "max_p": 0.7105638980865479, "max_p_per_token": [0.36693981289863586, 0.8540281057357788, 0.9977801442146301, 0.57196044921875, 0.756469190120697, 0.9381968975067139, 0.40334033966064453, 0.6311332583427429, 0.8503479957580566, 0.13627395033836365, 0.6873970627784729, 0.7549339532852173, 0.4647582173347473, 0.9999986886978149, 0.9997798800468445, 0.7548880577087402, 0.878114640712738, 0.9994680285453796, 0.16561181843280792, 0.9998568296432495], "n_positions_probed": 1, "per_restart_best": [4.953085422515869]}
+{"step": 285, "discrete_loss": 5.331405162811279, "best_sample_loss": 5.009186744689941, "soft_loss": 2.821164846420288, "best_discrete": 4.953085422515869, "best_soft": 2.821164846420288, "best_argmax": 5.331405162811279, "best_sampling": 4.953085422515869, "relax_gap": 0.47084028313978815, "n_match": 8, "g_first_norm": 137.3347625732422, "vocab_size": 50257, "entropy": 0.816215991973877, "entropy_per_token": [1.4861836433410645, 0.5204720497131348, 0.017721345648169518, 0.7167030572891235, 0.881873369216919, 0.30286362767219543, 1.207440972328186, 0.6607365608215332, 0.5074759721755981, 3.0547869205474854, 0.6643289923667908, 0.5618112087249756, 1.3381116390228271, 2.1125746570760384e-05, 0.0022342221345752478, 0.866341233253479, 0.4993950128555298, 0.004416176583617926, 3.029778003692627, 0.001625095377676189], "max_p": 0.7092054486274719, "max_p_per_token": [0.3642827570438385, 0.8499624133110046, 0.9977571368217468, 0.5668125152587891, 0.7516257166862488, 0.9331836700439453, 0.3994394838809967, 0.6303612589836121, 0.8525547981262207, 0.1319703906774521, 0.6913772821426392, 0.7548186779022217, 0.465286523103714, 0.9999986886978149, 0.9997877478599548, 0.7503274083137512, 0.8802995085716248, 0.9994847774505615, 0.16492250561714172, 0.9998551607131958], "n_positions_probed": 1, "per_restart_best": [4.953085422515869]}
+{"step": 286, "discrete_loss": 5.331405162811279, "best_sample_loss": 5.067626476287842, "soft_loss": 2.806710720062256, "best_discrete": 4.953085422515869, "best_soft": 2.806710720062256, "best_argmax": 5.331405162811279, "best_sampling": 4.953085422515869, "relax_gap": 0.47355141199168177, "n_match": 8, "g_first_norm": 137.45750427246094, "vocab_size": 50257, "entropy": 0.8199501037597656, "entropy_per_token": [1.494794249534607, 0.5300589203834534, 0.017869776114821434, 0.7187583446502686, 0.8933783173561096, 0.3147514760494232, 1.2284438610076904, 0.6611452102661133, 0.5004805326461792, 3.0760226249694824, 0.6602671146392822, 0.561531126499176, 1.3430490493774414, 2.0325000150478445e-05, 0.002166937803849578, 0.8807718753814697, 0.4951447546482086, 0.004290402866899967, 3.014409065246582, 0.0016468813410028815], "max_p": 0.7079140543937683, "max_p_per_token": [0.36146336793899536, 0.8457167148590088, 0.9977322816848755, 0.5612190961837769, 0.7467382550239563, 0.9291985034942627, 0.3916000425815582, 0.6295626759529114, 0.8546847105026245, 0.12738893926143646, 0.694735050201416, 0.755088210105896, 0.46572092175483704, 0.9999988079071045, 0.999795138835907, 0.7454777359962463, 0.8821456432342529, 0.9995013475418091, 0.1706603616476059, 0.9998533725738525], "n_positions_probed": 1, "per_restart_best": [4.953085422515869]}
+{"step": 287, "discrete_loss": 5.331405162811279, "best_sample_loss": 5.101622104644775, "soft_loss": 2.7930922508239746, "best_discrete": 4.953085422515869, "best_soft": 2.7930922508239746, "best_argmax": 5.331405162811279, "best_sampling": 4.953085422515869, "relax_gap": 0.4761057984662412, "n_match": 8, "g_first_norm": 137.0527801513672, "vocab_size": 50257, "entropy": 0.8230487704277039, "entropy_per_token": [1.5037930011749268, 0.5388193130493164, 0.017999975010752678, 0.7208125591278076, 0.9056458473205566, 0.3264523148536682, 1.2233054637908936, 0.6629709005355835, 0.49408242106437683, 3.0952439308166504, 0.6564801335334778, 0.5615264177322388, 1.3481727838516235, 1.9672583221108653e-05, 0.0021004383452236652, 0.8951058387756348, 0.49148494005203247, 0.004159808624535799, 3.0111300945281982, 0.0016697419341653585], "max_p": 0.7064703106880188, "max_p_per_token": [0.3579937219619751, 0.8417600989341736, 0.9977097511291504, 0.55495685338974, 0.7416728138923645, 0.9251561164855957, 0.39015278220176697, 0.6292568445205688, 0.8566041588783264, 0.12336438149213791, 0.6978395581245422, 0.7551130056381226, 0.4660022556781769, 0.9999988079071045, 0.99980229139328, 0.7406289577484131, 0.8837308883666992, 0.9995185136795044, 0.16829244792461395, 0.9998514652252197], "n_positions_probed": 1, "per_restart_best": [4.953085422515869]}
+{"step": 288, "discrete_loss": 5.331405162811279, "best_sample_loss": 4.991424083709717, "soft_loss": 2.7789015769958496, "best_discrete": 4.953085422515869, "best_soft": 2.7789015769958496, "best_argmax": 5.331405162811279, "best_sampling": 4.953085422515869, "relax_gap": 0.4787675121035972, "n_match": 8, "g_first_norm": 137.09246826171875, "vocab_size": 50257, "entropy": 0.825321614742279, "entropy_per_token": [1.5125672817230225, 0.5474116206169128, 0.018131406977772713, 0.7228479385375977, 0.9172513484954834, 0.33787479996681213, 1.2189874649047852, 0.6630573272705078, 0.4859824478626251, 3.1128275394439697, 0.6531701683998108, 0.5609583258628845, 1.3529051542282104, 1.906858778966125e-05, 0.00203788373619318, 0.9101017713546753, 0.48854535818099976, 0.004030273761600256, 2.9960312843322754, 0.0016930929850786924], "max_p": 0.7053826451301575, "max_p_per_token": [0.3544532060623169, 0.8377960324287415, 0.9976872205734253, 0.5482708215713501, 0.7367324829101562, 0.921066164970398, 0.3871353268623352, 0.6289718151092529, 0.8587282299995422, 0.11926872283220291, 0.7004546523094177, 0.7556262016296387, 0.4664934277534485, 0.9999988079071045, 0.9998090863227844, 0.7354335188865662, 0.8850164413452148, 0.9995354413986206, 0.175325408577919, 0.9998495578765869], "n_positions_probed": 1, "per_restart_best": [4.953085422515869]}
+{"step": 289, "discrete_loss": 5.474322319030762, "best_sample_loss": 5.3890228271484375, "soft_loss": 2.7653648853302, "best_discrete": 4.953085422515869, "best_soft": 2.7653648853302, "best_argmax": 5.331405162811279, "best_sampling": 4.953085422515869, "relax_gap": 0.4948479968530949, "n_match": 8, "g_first_norm": 136.51051330566406, "vocab_size": 50257, "entropy": 0.7072558403015137, "entropy_per_token": [1.5216801166534424, 0.5544945001602173, 0.0182491447776556, 0.7248046398162842, 0.9291276931762695, 0.3487597703933716, 1.2144665718078613, 0.6626068353652954, 0.4799457788467407, 0.7059072852134705, 0.6498578786849976, 0.5607404708862305, 1.357985258102417, 1.836349292716477e-05, 0.0019751761574298143, 0.9242782592773438, 0.486211895942688, 0.003898880910128355, 2.9983901977539062, 0.0017170589417219162], "max_p": 0.7422398328781128, "max_p_per_token": [0.35044756531715393, 0.8344663381576538, 0.997666597366333, 0.5409891605377197, 0.7318469285964966, 0.9170272946357727, 0.3852420449256897, 0.6296982765197754, 0.8605946898460388, 0.8855369091033936, 0.7030870914459229, 0.7558297514915466, 0.4665202498435974, 0.999998927116394, 0.9998158812522888, 0.7305296659469604, 0.8860510587692261, 0.9995524287223816, 0.17004793882369995, 0.9998476505279541], "n_positions_probed": 1, "per_restart_best": [4.953085422515869]}
+{"step": 290, "discrete_loss": 5.474322319030762, "best_sample_loss": 5.120504856109619, "soft_loss": 4.401832580566406, "best_discrete": 4.953085422515869, "best_soft": 2.7653648853302, "best_argmax": 5.331405162811279, "best_sampling": 4.953085422515869, "relax_gap": 0.1959127862705464, "n_match": 8, "g_first_norm": 129.56707763671875, "vocab_size": 50257, "entropy": 0.7098935842514038, "entropy_per_token": [1.5418611764907837, 0.5380915403366089, 0.018224483355879784, 0.7298628091812134, 0.9211010336875916, 0.3594713807106018, 1.2243797779083252, 0.6689639091491699, 0.47477003931999207, 0.6952654123306274, 0.7454996109008789, 0.5925285816192627, 1.391406536102295, 1.8716031263465993e-05, 0.002023351611569524, 0.9511529207229614, 0.48844337463378906, 0.003786348504945636, 2.8492751121520996, 0.0017453781329095364], "max_p": 0.7381974458694458, "max_p_per_token": [0.33707138895988464, 0.8432359099388123, 0.9976715445518494, 0.5117067694664001, 0.735855221748352, 0.9127811193466187, 0.35993626713752747, 0.6171123385429382, 0.8632689714431763, 0.8875819444656372, 0.6660352945327759, 0.7262817025184631, 0.44639483094215393, 0.999998927116394, 0.9998109936714172, 0.7197152972221375, 0.8858194351196289, 0.9995669722557068, 0.25425881147384644, 0.999845027923584], "n_positions_probed": 1, "per_restart_best": [4.953085422515869]}
+{"step": 291, "discrete_loss": 5.349789142608643, "best_sample_loss": 5.053633213043213, "soft_loss": 4.362703800201416, "best_discrete": 4.953085422515869, "best_soft": 2.7653648853302, "best_argmax": 5.331405162811279, "best_sampling": 4.953085422515869, "relax_gap": 0.1845092051470851, "n_match": 7, "g_first_norm": 122.32888793945312, "vocab_size": 50257, "entropy": 0.7204559445381165, "entropy_per_token": [1.5608388185501099, 0.518413782119751, 0.018371500074863434, 0.731654167175293, 0.9267265200614929, 0.3708922266960144, 1.2197370529174805, 0.6716800928115845, 0.47191545367240906, 0.6828281879425049, 0.7652304768562317, 0.6138487458229065, 1.4131126403808594, 1.8536782590672374e-05, 0.002011245349422097, 0.957464337348938, 0.4871840476989746, 0.003601066768169403, 2.9917945861816406, 0.001795948832295835], "max_p": 0.7303640246391296, "max_p_per_token": [0.32324907183647156, 0.8530246019363403, 0.9976546168327332, 0.5082460045814514, 0.7353864312171936, 0.9081795811653137, 0.36764588952064514, 0.6112865805625916, 0.8648192286491394, 0.8899773359298706, 0.6417844891548157, 0.703578770160675, 0.4341512620449066, 0.999998927116394, 0.9998125433921814, 0.7177479863166809, 0.8865709900856018, 0.9995908141136169, 0.1647351086139679, 0.9998400211334229], "n_positions_probed": 1, "per_restart_best": [4.953085422515869]}
+{"step": 292, "discrete_loss": 5.349789142608643, "best_sample_loss": 5.094767093658447, "soft_loss": 4.296204090118408, "best_discrete": 4.953085422515869, "best_soft": 2.7653648853302, "best_argmax": 5.331405162811279, "best_sampling": 4.953085422515869, "relax_gap": 0.19693954741111339, "n_match": 7, "g_first_norm": 134.88319396972656, "vocab_size": 50257, "entropy": 0.7213045954704285, "entropy_per_token": [1.5665181875228882, 0.5107157826423645, 0.018514102324843407, 0.7286410331726074, 0.916151762008667, 0.3860967755317688, 1.218941569328308, 0.6752783060073853, 0.4665622115135193, 0.6649341583251953, 0.7987103462219238, 0.6357138752937317, 1.4927159547805786, 1.874898953246884e-05, 0.0020432129967957735, 0.9865521192550659, 0.4942135810852051, 0.0034203564282506704, 2.8585309982299805, 0.0018179729813709855], "max_p": 0.7323052287101746, "max_p_per_token": [0.30987125635147095, 0.8574228286743164, 0.9976352453231812, 0.5493803024291992, 0.74032062292099, 0.9019221067428589, 0.39161214232444763, 0.6030369997024536, 0.8672293424606323, 0.8933872580528259, 0.5938865542411804, 0.6768547892570496, 0.4262220859527588, 0.999998927116394, 0.9998094439506531, 0.7057047486305237, 0.8847324848175049, 0.9996139407157898, 0.2476254105567932, 0.99983811378479], "n_positions_probed": 1, "per_restart_best": [4.953085422515869]}
+{"step": 293, "discrete_loss": 5.349789142608643, "best_sample_loss": 5.451720714569092, "soft_loss": 4.233363151550293, "best_discrete": 4.953085422515869, "best_soft": 2.7653648853302, "best_argmax": 5.331405162811279, "best_sampling": 4.953085422515869, "relax_gap": 0.2086859801943451, "n_match": 7, "g_first_norm": 138.33987426757812, "vocab_size": 50257, "entropy": 0.7311789393424988, "entropy_per_token": [1.5708264112472534, 0.49733683466911316, 0.01865459233522415, 0.7191838026046753, 0.9264032244682312, 0.3983539044857025, 1.210928201675415, 0.6765952110290527, 0.46215444803237915, 0.6471037864685059, 0.8501490950584412, 0.6837911605834961, 1.478994369506836, 0.0011018447112292051, 0.002022040542215109, 0.9920638799667358, 0.4955386519432068, 0.0032186575699597597, 2.9873006343841553, 0.0018581498879939318], "max_p": 0.7226872444152832, "max_p_per_token": [0.29406067728996277, 0.8638902306556702, 0.9976192116737366, 0.5879576206207275, 0.737228274345398, 0.8964899778366089, 0.4123974144458771, 0.5998261570930481, 0.8691937327384949, 0.8967634439468384, 0.5181226134300232, 0.5929979085922241, 0.4367428421974182, 0.999891996383667, 0.9998119473457336, 0.7041813135147095, 0.884528398513794, 0.9996395111083984, 0.16256798803806305, 0.9998342990875244], "n_positions_probed": 1, "per_restart_best": [4.953085422515869]}
+{"step": 294, "discrete_loss": 5.365499973297119, "best_sample_loss": 6.021230220794678, "soft_loss": 3.9908695220947266, "best_discrete": 4.953085422515869, "best_soft": 2.7653648853302, "best_argmax": 5.331405162811279, "best_sampling": 4.953085422515869, "relax_gap": 0.25619801659558616, "n_match": 8, "g_first_norm": 193.50379943847656, "vocab_size": 50257, "entropy": 0.7371012568473816, "entropy_per_token": [1.5599513053894043, 0.4994944930076599, 0.018893670290708542, 0.6991851925849915, 0.9183578491210938, 0.4122954308986664, 1.2180949449539185, 0.6800594329833984, 0.4572913944721222, 0.6231465339660645, 0.9683064222335815, 0.6986969709396362, 1.427764654159546, 0.0011026242282241583, 0.1085008755326271, 1.0175347328186035, 0.5044812560081482, 0.0030077442061156034, 2.9239702224731445, 0.0018896381370723248], "max_p": 0.7280293703079224, "max_p_per_token": [0.32682541012763977, 0.8639312386512756, 0.9975844621658325, 0.6349610090255737, 0.7412869334220886, 0.8900380730628967, 0.40377241373062134, 0.5909001231193542, 0.8709369897842407, 0.9011811017990112, 0.5579503774642944, 0.5425286889076233, 0.4763818383216858, 0.9998918771743774, 0.9791930913925171, 0.6934833526611328, 0.8819242119789124, 0.9996659755706787, 0.20831765234470367, 0.9998314380645752], "n_positions_probed": 1, "per_restart_best": [4.953085422515869]}
+{"step": 295, "discrete_loss": 6.317814350128174, "best_sample_loss": 5.650676727294922, "soft_loss": 3.841175079345703, "best_discrete": 4.953085422515869, "best_soft": 2.7653648853302, "best_argmax": 5.331405162811279, "best_sampling": 4.953085422515869, "relax_gap": 0.39200887103183485, "n_match": 7, "g_first_norm": 144.2630157470703, "vocab_size": 50257, "entropy": 0.7426545023918152, "entropy_per_token": [1.545762062072754, 0.49984219670295715, 0.01912504993379116, 0.6953004598617554, 0.9270902276039124, 0.42147111892700195, 1.2218701839447021, 0.6825710535049438, 0.4570881724357605, 0.6011238098144531, 0.9887599945068359, 0.6987866163253784, 1.423789143562317, 0.0010904254158958793, 0.1042165532708168, 1.0617293119430542, 0.5110538005828857, 0.0028287838213145733, 2.9876298904418945, 0.001960420748218894], "max_p": 0.7177548408508301, "max_p_per_token": [0.3473130464553833, 0.8644406199455261, 0.9975516200065613, 0.6450269818305969, 0.7403609752655029, 0.8857321739196777, 0.38476964831352234, 0.5836774110794067, 0.8707993030548096, 0.9053389430046082, 0.5279408097267151, 0.5454714298248291, 0.4838089048862457, 0.9998931884765625, 0.9802345633506775, 0.5549392104148865, 0.8797975182533264, 0.9996882677078247, 0.15848736464977264, 0.999824583530426], "n_positions_probed": 1, "per_restart_best": [4.953085422515869]}
+{"step": 296, "discrete_loss": 5.365499973297119, "best_sample_loss": 5.539453983306885, "soft_loss": 4.432198524475098, "best_discrete": 4.953085422515869, "best_soft": 2.7653648853302, "best_argmax": 5.331405162811279, "best_sampling": 4.953085422515869, "relax_gap": 0.17394491724291342, "n_match": 8, "g_first_norm": 319.7857666015625, "vocab_size": 50257, "entropy": 0.7338671088218689, "entropy_per_token": [1.504129409790039, 0.5231064558029175, 0.01938435062766075, 0.7019243240356445, 0.9594708681106567, 0.45482897758483887, 1.2168498039245605, 0.6819154024124146, 0.4664471745491028, 0.5944415330886841, 1.002028465270996, 0.6961257457733154, 1.4071574211120605, 0.0010553350439295173, 0.09619801491498947, 0.9012104868888855, 0.5440727472305298, 0.0025999273639172316, 2.9024643898010254, 0.0019322875887155533], "max_p": 0.717958390712738, "max_p_per_token": [0.3733513355255127, 0.8540729284286499, 0.9975196719169617, 0.6423669457435608, 0.7260705232620239, 0.8713082075119019, 0.37588730454444885, 0.5854261517524719, 0.8662471771240234, 0.906589150428772, 0.5140936970710754, 0.5620222687721252, 0.49960193037986755, 0.9998970031738281, 0.9821043610572815, 0.5048811435699463, 0.8692089915275574, 0.9997163414955139, 0.2289741039276123, 0.9998273849487305], "n_positions_probed": 1, "per_restart_best": [4.953085422515869]}
+{"step": 297, "discrete_loss": 6.251962661743164, "best_sample_loss": 5.968542098999023, "soft_loss": 4.065258979797363, "best_discrete": 4.953085422515869, "best_soft": 2.7653648853302, "best_argmax": 5.331405162811279, "best_sampling": 4.953085422515869, "relax_gap": 0.34976275455492034, "n_match": 8, "g_first_norm": 149.2835693359375, "vocab_size": 50257, "entropy": 0.737152636051178, "entropy_per_token": [1.494264841079712, 0.5268675088882446, 0.019686277955770493, 0.69989013671875, 0.9997943639755249, 0.4642890989780426, 1.2079733610153198, 0.6819782257080078, 0.46319475769996643, 0.5877079963684082, 1.0202069282531738, 0.7017119526863098, 1.4129149913787842, 0.0010579340159893036, 0.09008719027042389, 0.7864727973937988, 0.553413450717926, 0.0024344241246581078, 3.0271859169006348, 0.0019214354688301682], "max_p": 0.7222448587417603, "max_p_per_token": [0.38690489530563354, 0.8526764512062073, 0.9974796175956726, 0.6499228477478027, 0.7093815207481384, 0.8663596510887146, 0.3920869529247284, 0.585165798664093, 0.8675887584686279, 0.9077942371368408, 0.5102964043617249, 0.5413573980331421, 0.49713316559791565, 0.999896764755249, 0.9835017323493958, 0.6910133957862854, 0.8661483526229858, 0.9997366070747375, 0.1406235545873642, 0.999828577041626], "n_positions_probed": 1, "per_restart_best": [4.953085422515869]}
+{"step": 298, "discrete_loss": 6.413926601409912, "best_sample_loss": 5.124831676483154, "soft_loss": 3.8738136291503906, "best_discrete": 4.953085422515869, "best_soft": 2.7653648853302, "best_argmax": 5.331405162811279, "best_sampling": 4.953085422515869, "relax_gap": 0.3960308762655863, "n_match": 8, "g_first_norm": 150.4010467529297, "vocab_size": 50257, "entropy": 0.6621589064598083, "entropy_per_token": [1.4831236600875854, 0.5373992919921875, 0.020190199837088585, 0.6994524002075195, 0.9830753207206726, 0.4706329107284546, 1.2040421962738037, 0.6843540668487549, 0.4657982289791107, 0.5690677165985107, 1.04714035987854, 0.698736310005188, 1.4067561626434326, 0.0010601039975881577, 0.08838005363941193, 0.7551765441894531, 0.559833824634552, 0.002293266821652651, 1.5646734237670898, 0.0019908342510461807], "max_p": 0.7499737739562988, "max_p_per_token": [0.39929673075675964, 0.8488354682922363, 0.997404158115387, 0.6559812426567078, 0.7195611596107483, 0.8624131083488464, 0.38585785031318665, 0.5779649019241333, 0.8660567402839661, 0.9111361503601074, 0.495716392993927, 0.5596725344657898, 0.5028270483016968, 0.9998965263366699, 0.9839105606079102, 0.7303398847579956, 0.8636813759803772, 0.9997536540031433, 0.6393479704856873, 0.9998219609260559], "n_positions_probed": 1, "per_restart_best": [4.953085422515869]}
+{"step": 299, "discrete_loss": 6.413926601409912, "best_sample_loss": 5.4305925369262695, "soft_loss": 5.4742751121521, "best_discrete": 4.953085422515869, "best_soft": 2.7653648853302, "best_argmax": 5.331405162811279, "best_sampling": 4.953085422515869, "relax_gap": 0.1465017527720473, "n_match": 8, "g_first_norm": 146.9151611328125, "vocab_size": 50257, "entropy": 0.6808944940567017, "entropy_per_token": [1.4721128940582275, 0.5468183755874634, 0.020517606288194656, 0.6960068941116333, 1.012049674987793, 0.4702882170677185, 1.194222331047058, 0.6884061098098755, 0.47785311937332153, 0.5869677066802979, 1.0906764268875122, 0.707662045955658, 1.4457042217254639, 0.0009986123768612742, 0.09045156836509705, 0.7385643720626831, 0.5915707349777222, 0.002140138065442443, 1.7793362140655518, 0.0055419872514903545], "max_p": 0.7419423460960388, "max_p_per_token": [0.4145202338695526, 0.8443877696990967, 0.9973575472831726, 0.6623108386993408, 0.7128131985664368, 0.8610753417015076, 0.4137110412120819, 0.5640272498130798, 0.860539436340332, 0.9075215458869934, 0.4906577169895172, 0.5018065571784973, 0.48404353857040405, 0.9999030828475952, 0.9834613800048828, 0.7504996657371521, 0.8524868488311768, 0.9997720122337341, 0.5385269522666931, 0.9994242191314697], "n_positions_probed": 1, "per_restart_best": [4.953085422515869]}
+{"step": 300, "discrete_loss": 5.26613187789917, "best_sample_loss": 4.966040134429932, "soft_loss": 5.102934837341309, "best_discrete": 4.953085422515869, "best_soft": 2.7653648853302, "best_argmax": 5.26613187789917, "best_sampling": 4.953085422515869, "relax_gap": 0.030989926637189705, "n_match": 9, "g_first_norm": 211.80384826660156, "vocab_size": 50257, "entropy": 0.6439313888549805, "entropy_per_token": [0.1981886625289917, 0.5469022393226624, 0.020840583369135857, 0.7034400105476379, 1.0390465259552002, 0.4670906066894531, 1.1873654127120972, 0.6908026933670044, 0.4868408143520355, 0.5945284366607666, 1.1026437282562256, 0.7074475884437561, 1.4555180072784424, 0.0009454190148971975, 0.09109089523553848, 0.7148387432098389, 0.617997407913208, 0.0020112483762204647, 2.2449545860290527, 0.006134605035185814], "max_p": 0.7559213042259216, "max_p_per_token": [0.9638904929161072, 0.8436732888221741, 0.9973095655441284, 0.652874231338501, 0.7074708342552185, 0.8611502051353455, 0.42838677763938904, 0.5540668368339539, 0.8560600876808167, 0.9058036804199219, 0.4372388422489166, 0.5308322906494141, 0.4788081645965576, 0.9999088048934937, 0.98334139585495, 0.7715501189231873, 0.8429971933364868, 0.9997872710227966, 0.30392324924468994, 0.9993526339530945], "n_positions_probed": 1, "per_restart_best": [4.953085422515869]}
+{"step": 301, "discrete_loss": 5.101364612579346, "best_sample_loss": 4.97357177734375, "soft_loss": 4.12177848815918, "best_discrete": 4.953085422515869, "best_soft": 2.7653648853302, "best_argmax": 5.101364612579346, "best_sampling": 4.953085422515869, "relax_gap": 0.19202433051043355, "n_match": 9, "g_first_norm": 167.05526733398438, "vocab_size": 50257, "entropy": 0.6586177945137024, "entropy_per_token": [0.2283879965543747, 0.6145323514938354, 0.021481644362211227, 0.7024455666542053, 1.0291752815246582, 0.474296510219574, 1.1833739280700684, 0.6901168823242188, 0.4819697141647339, 0.6002821922302246, 1.1709405183792114, 0.7091174721717834, 1.4819536209106445, 0.0009445958421565592, 0.0885002389550209, 0.7282376289367676, 0.6133290529251099, 0.0018423879519104958, 2.3449363708496094, 0.0064909690991044044], "max_p": 0.7534187436103821, "max_p_per_token": [0.9570518136024475, 0.7499021291732788, 0.9972071051597595, 0.6566246151924133, 0.7174122333526611, 0.8572922348976135, 0.41767579317092896, 0.5568556189537048, 0.8582987189292908, 0.904532790184021, 0.4940284490585327, 0.5309565663337708, 0.45316386222839355, 0.9999089241027832, 0.9839471578598022, 0.7695509195327759, 0.8442310094833374, 0.9998071789741516, 0.3206196129322052, 0.9993088245391846], "n_positions_probed": 1, "per_restart_best": [4.953085422515869]}
+{"step": 302, "discrete_loss": 5.101364612579346, "best_sample_loss": 4.959473133087158, "soft_loss": 3.9882194995880127, "best_discrete": 4.953085422515869, "best_soft": 2.7653648853302, "best_argmax": 5.101364612579346, "best_sampling": 4.953085422515869, "relax_gap": 0.21820536219788178, "n_match": 9, "g_first_norm": 130.06954956054688, "vocab_size": 50257, "entropy": 0.6755321621894836, "entropy_per_token": [0.2519438564777374, 0.6280801296234131, 0.18090926110744476, 0.7022871971130371, 1.0750291347503662, 0.479102224111557, 1.166205644607544, 0.6906569004058838, 0.4740730822086334, 0.5976059436798096, 1.1821738481521606, 0.7086794376373291, 1.4906558990478516, 0.0009661235962994397, 0.08350011706352234, 0.7301431894302368, 0.6012892723083496, 0.0017234630649909377, 2.458702564239502, 0.00691565778106451], "max_p": 0.7464107275009155, "max_p_per_token": [0.9515517354011536, 0.7388285994529724, 0.9570515155792236, 0.6590545773506165, 0.7033742070198059, 0.8546162247657776, 0.43977031111717224, 0.5543715953826904, 0.8622406125068665, 0.904940664768219, 0.4633309543132782, 0.5416303873062134, 0.4325847029685974, 0.9999066591262817, 0.9850767254829407, 0.7744283676147461, 0.847579836845398, 0.9998210072517395, 0.25879883766174316, 0.9992563128471375], "n_positions_probed": 1, "per_restart_best": [4.953085422515869]}
+{"step": 303, "discrete_loss": 5.101364612579346, "best_sample_loss": 5.001049518585205, "soft_loss": 3.9180920124053955, "best_discrete": 4.953085422515869, "best_soft": 2.7653648853302, "best_argmax": 5.101364612579346, "best_sampling": 4.953085422515869, "relax_gap": 0.23195217163190876, "n_match": 9, "g_first_norm": 130.88479614257812, "vocab_size": 50257, "entropy": 0.6623749732971191, "entropy_per_token": [0.2767464220523834, 0.641645073890686, 0.18325524032115936, 0.3479316234588623, 1.1048789024353027, 0.48481687903404236, 1.1596986055374146, 0.691554069519043, 0.4667869806289673, 0.5910925269126892, 1.1885991096496582, 0.7099073529243469, 1.483036994934082, 0.0009901742450892925, 0.07927463948726654, 0.7404939532279968, 0.5918906927108765, 0.0016089007258415222, 2.4959654808044434, 0.007326256949454546], "max_p": 0.7555515170097351, "max_p_per_token": [0.9455975890159607, 0.7267806529998779, 0.9562912583351135, 0.8981198072433472, 0.6953703165054321, 0.851628839969635, 0.43270009756088257, 0.5500538349151611, 0.8657769560813904, 0.9060543179512024, 0.4301862120628357, 0.5424240231513977, 0.42287760972976685, 0.9999040365219116, 0.9860206246376038, 0.7743876576423645, 0.849992036819458, 0.9998341798782349, 0.277824729681015, 0.9992049336433411], "n_positions_probed": 1, "per_restart_best": [4.953085422515869]}
+{"step": 304, "discrete_loss": 5.345571994781494, "best_sample_loss": 5.20694637298584, "soft_loss": 3.8855316638946533, "best_discrete": 4.953085422515869, "best_soft": 2.7653648853302, "best_argmax": 5.101364612579346, "best_sampling": 4.953085422515869, "relax_gap": 0.2731307954157526, "n_match": 8, "g_first_norm": 132.58380126953125, "vocab_size": 50257, "entropy": 0.6129048466682434, "entropy_per_token": [0.2979963421821594, 0.6516677141189575, 0.1855318695306778, 0.364623486995697, 0.026429710909724236, 0.4883359670639038, 1.1457383632659912, 0.6923955678939819, 0.45952722430229187, 0.5841450691223145, 1.2052276134490967, 0.7129242420196533, 1.47561776638031, 0.0010215420043095946, 0.07459904253482819, 0.74899822473526, 0.5844486951828003, 0.0015162109630182385, 2.549592971801758, 0.007759082596749067], "max_p": 0.7660495638847351, "max_p_per_token": [0.9403301477432251, 0.7175946831703186, 0.9555587768554688, 0.8907887935638428, 0.9965338706970215, 0.8496190309524536, 0.4334604740142822, 0.5456647872924805, 0.869269609451294, 0.90723717212677, 0.4091433882713318, 0.5318841934204102, 0.41390153765678406, 0.9999006986618042, 0.9870412349700928, 0.7752697467803955, 0.851813018321991, 0.9998447895050049, 0.24698419868946075, 0.9991501569747925], "n_positions_probed": 1, "per_restart_best": [4.953085422515869]}
+{"step": 305, "discrete_loss": 5.414502143859863, "best_sample_loss": 5.049391269683838, "soft_loss": 4.094637870788574, "best_discrete": 4.953085422515869, "best_soft": 2.7653648853302, "best_argmax": 5.101364612579346, "best_sampling": 4.953085422515869, "relax_gap": 0.24376465979758405, "n_match": 7, "g_first_norm": 143.7354278564453, "vocab_size": 50257, "entropy": 0.6169902086257935, "entropy_per_token": [0.3344465494155884, 0.6559814810752869, 0.18651290237903595, 0.3841956555843353, 0.028130345046520233, 0.5404220819473267, 1.1369249820709229, 0.6938127279281616, 0.4411897659301758, 0.5618605613708496, 1.2275582551956177, 0.7060012221336365, 1.4692904949188232, 0.0010005139047279954, 0.07264159619808197, 0.7622684836387634, 0.575792133808136, 0.0013988650171086192, 2.551771879196167, 0.008602715097367764], "max_p": 0.7666053175926208, "max_p_per_token": [0.9310992956161499, 0.7148897051811218, 0.9552074670791626, 0.8821161389350891, 0.9962742328643799, 0.8385630249977112, 0.43943920731544495, 0.5376630425453186, 0.8776168823242188, 0.9114263653755188, 0.3973352313041687, 0.5780462026596069, 0.3943054974079132, 0.9999029636383057, 0.987478494644165, 0.7729449272155762, 0.8542342782020569, 0.9998581409454346, 0.2646617889404297, 0.9990425705909729], "n_positions_probed": 1, "per_restart_best": [4.953085422515869]}
+{"step": 306, "discrete_loss": 5.414502143859863, "best_sample_loss": 5.0871500968933105, "soft_loss": 4.0425705909729, "best_discrete": 4.953085422515869, "best_soft": 2.7653648853302, "best_argmax": 5.101364612579346, "best_sampling": 4.953085422515869, "relax_gap": 0.2533809233860506, "n_match": 7, "g_first_norm": 138.98828125, "vocab_size": 50257, "entropy": 0.6206359267234802, "entropy_per_token": [0.3709447383880615, 0.6606531143188477, 0.1875033676624298, 0.4051767587661743, 0.0296761617064476, 0.5472050905227661, 1.1268709897994995, 0.694823145866394, 0.42556530237197876, 0.5449548363685608, 1.2635400295257568, 0.7006357908248901, 1.455315113067627, 0.0009859215933829546, 0.07020562887191772, 0.7750487327575684, 0.5686745643615723, 0.001296902890317142, 2.5741519927978516, 0.009489341638982296], "max_p": 0.7650755047798157, "max_p_per_token": [0.9214696288108826, 0.7118639349937439, 0.9548580050468445, 0.8724075555801392, 0.9960364699363708, 0.8347441554069519, 0.448439359664917, 0.5308738946914673, 0.8845354914665222, 0.9145600199699402, 0.38076332211494446, 0.6031010150909424, 0.3804471790790558, 0.9999045133590698, 0.988008975982666, 0.7710777521133423, 0.8561128377914429, 0.9998695850372314, 0.253507524728775, 0.9989277720451355], "n_positions_probed": 1, "per_restart_best": [4.953085422515869]}
+{"step": 307, "discrete_loss": 5.414502143859863, "best_sample_loss": 4.937316417694092, "soft_loss": 4.008044242858887, "best_discrete": 4.937316417694092, "best_soft": 2.7653648853302, "best_argmax": 5.101364612579346, "best_sampling": 4.937316417694092, "relax_gap": 0.2597575665559434, "n_match": 7, "g_first_norm": 138.2628631591797, "vocab_size": 50257, "entropy": 0.6203514933586121, "entropy_per_token": [0.4098604619503021, 0.6642544269561768, 0.18868452310562134, 0.42487427592277527, 0.03125585615634918, 0.5526524186134338, 1.1195168495178223, 0.6513010263442993, 0.4107479453086853, 0.5277483463287354, 1.279712438583374, 0.6965889930725098, 1.4342329502105713, 0.000973545596934855, 0.06793492287397385, 0.789581298828125, 0.5628786087036133, 0.00120157515630126, 2.5825557708740234, 0.010474168695509434], "max_p": 0.7717965245246887, "max_p_per_token": [0.9108670353889465, 0.7099825143814087, 0.9544501304626465, 0.8629859089851379, 0.9957913160324097, 0.8314882516860962, 0.4519568383693695, 0.6487470269203186, 0.8909090161323547, 0.9177265763282776, 0.39789119362831116, 0.6203840374946594, 0.3723883032798767, 0.9999058246612549, 0.9884999394416809, 0.7684239149093628, 0.8575042486190796, 0.9998801946640015, 0.25735026597976685, 0.9987977743148804], "n_positions_probed": 1, "per_restart_best": [4.937316417694092]}
+{"step": 308, "discrete_loss": 5.389908790588379, "best_sample_loss": 4.987206935882568, "soft_loss": 3.9948601722717285, "best_discrete": 4.937316417694092, "best_soft": 2.7653648853302, "best_argmax": 5.101364612579346, "best_sampling": 4.937316417694092, "relax_gap": 0.2588260158970821, "n_match": 8, "g_first_norm": 135.7635498046875, "vocab_size": 50257, "entropy": 0.6354944109916687, "entropy_per_token": [0.4494907259941101, 0.6688734292984009, 0.19042079150676727, 0.44413548707962036, 0.032863300293684006, 0.559764564037323, 1.1131017208099365, 0.6607747077941895, 0.6239846348762512, 0.5122677087783813, 1.3066930770874023, 0.6974734663963318, 1.4121859073638916, 0.0009619826450943947, 0.06568355858325958, 0.8060864210128784, 0.5568618774414062, 0.0011165746254846454, 2.5956177711486816, 0.01153053529560566], "max_p": 0.764655351638794, "max_p_per_token": [0.899623453617096, 0.7070230841636658, 0.9538625478744507, 0.8534419536590576, 0.9955413937568665, 0.8274105787277222, 0.4569533169269562, 0.6325091123580933, 0.7917665839195251, 0.9205328822135925, 0.39124250411987305, 0.6274275779724121, 0.36919867992401123, 0.9999071359634399, 0.9889812469482422, 0.7648693323135376, 0.8590598106384277, 0.999889612197876, 0.2552104890346527, 0.9986560344696045], "n_positions_probed": 1, "per_restart_best": [4.937316417694092]}
+{"step": 309, "discrete_loss": 5.389908790588379, "best_sample_loss": 5.360085964202881, "soft_loss": 3.960477352142334, "best_discrete": 4.937316417694092, "best_soft": 2.7653648853302, "best_argmax": 5.101364612579346, "best_sampling": 4.937316417694092, "relax_gap": 0.26520512572347327, "n_match": 8, "g_first_norm": 135.9720458984375, "vocab_size": 50257, "entropy": 0.6673372387886047, "entropy_per_token": [0.49308356642723083, 0.6714817881584167, 0.19267858564853668, 0.46282634139060974, 0.03435587137937546, 0.565488338470459, 1.1098800897598267, 0.6687948107719421, 0.6302851438522339, 1.0425801277160645, 1.3226675987243652, 0.6997847557067871, 1.3883211612701416, 0.0009600340854376554, 0.06337548792362213, 0.8238723874092102, 0.5507180690765381, 0.001036164816468954, 2.6118690967559814, 0.01268431730568409], "max_p": 0.7533986568450928, "max_p_per_token": [0.8868311643600464, 0.7062647342681885, 0.9531075954437256, 0.8438436388969421, 0.9953076243400574, 0.824051022529602, 0.4569692313671112, 0.616935133934021, 0.7896364331245422, 0.7174547910690308, 0.40049225091934204, 0.6321857571601868, 0.3851619362831116, 0.999907374382019, 0.9894677996635437, 0.7607176303863525, 0.8606546521186829, 0.9998983144760132, 0.2505877912044525, 0.9984983205795288], "n_positions_probed": 1, "per_restart_best": [4.937316417694092]}
+{"step": 310, "discrete_loss": 5.389908790588379, "best_sample_loss": 4.934022426605225, "soft_loss": 3.9355149269104004, "best_discrete": 4.934022426605225, "best_soft": 2.7653648853302, "best_argmax": 5.101364612579346, "best_sampling": 4.934022426605225, "relax_gap": 0.2698364518185497, "n_match": 8, "g_first_norm": 137.77806091308594, "vocab_size": 50257, "entropy": 0.6725053787231445, "entropy_per_token": [0.5380605459213257, 0.6737128496170044, 0.19541707634925842, 0.48162829875946045, 0.035674065351486206, 0.5699441432952881, 1.1076271533966064, 0.6763836145401001, 0.6375261545181274, 1.0374658107757568, 1.3349759578704834, 0.7031165957450867, 1.3648738861083984, 0.0009591727866791189, 0.061351388692855835, 0.844300389289856, 0.5445048809051514, 0.0009585308143869042, 2.6276068687438965, 0.014019661583006382], "max_p": 0.7518449425697327, "max_p_per_token": [0.873176634311676, 0.7059462666511536, 0.9521886706352234, 0.8337766528129578, 0.9951000809669495, 0.8212183713912964, 0.4544316232204437, 0.5999011397361755, 0.787102222442627, 0.7165952920913696, 0.41243380308151245, 0.6360743641853333, 0.3980681598186493, 0.999907374382019, 0.9898922443389893, 0.7555094361305237, 0.8622413277626038, 0.9999067783355713, 0.2451157420873642, 0.9983127117156982], "n_positions_probed": 1, "per_restart_best": [4.934022426605225]}
+{"step": 311, "discrete_loss": 5.389908790588379, "best_sample_loss": 4.934022426605225, "soft_loss": 3.912968397140503, "best_discrete": 4.934022426605225, "best_soft": 2.7653648853302, "best_argmax": 5.101364612579346, "best_sampling": 4.934022426605225, "relax_gap": 0.27401955224675495, "n_match": 8, "g_first_norm": 137.4132843017578, "vocab_size": 50257, "entropy": 0.6431536078453064, "entropy_per_token": [0.5842708349227905, 0.6768522262573242, 0.1983097791671753, 0.500715434551239, 0.03698977828025818, 0.5744717121124268, 1.103562831878662, 0.6828190088272095, 0.6450040340423584, 1.0343191623687744, 1.356724500656128, 0.0043621608056128025, 1.3430650234222412, 0.0009611959685571492, 0.059362709522247314, 0.8664075136184692, 0.5401538014411926, 0.0008873154292814434, 2.6383891105651855, 0.015443607233464718], "max_p": 0.7674791216850281, "max_p_per_token": [0.8586064577102661, 0.7048046588897705, 0.9512137174606323, 0.8231468200683594, 0.9948921203613281, 0.818314254283905, 0.4550086259841919, 0.582708477973938, 0.7844734787940979, 0.7149542570114136, 0.4087957739830017, 0.9995015859603882, 0.40835967659950256, 0.9999071359634399, 0.9903049468994141, 0.7496312856674194, 0.8632457852363586, 0.9999144077301025, 0.2436871975660324, 0.9981110095977783], "n_positions_probed": 1, "per_restart_best": [4.934022426605225]}
+{"step": 312, "discrete_loss": 5.389908790588379, "best_sample_loss": 4.90839958190918, "soft_loss": 3.9187850952148438, "best_discrete": 4.90839958190918, "best_soft": 2.7653648853302, "best_argmax": 5.101364612579346, "best_sampling": 4.90839958190918, "relax_gap": 0.2729403692215249, "n_match": 7, "g_first_norm": 133.81350708007812, "vocab_size": 50257, "entropy": 0.6639610528945923, "entropy_per_token": [0.6267326474189758, 0.6798455715179443, 0.2021184116601944, 0.5198354721069336, 0.038335785269737244, 0.5765817165374756, 1.096196174621582, 0.6882123351097107, 0.6447376012802124, 1.0363775491714478, 1.3636893033981323, 0.004514486063271761, 1.6473857164382935, 0.0009622888173907995, 0.056617431342601776, 0.8919367790222168, 0.5423521995544434, 0.0008217698195949197, 2.6445140838623047, 0.017452886328101158], "max_p": 0.760807454586029, "max_p_per_token": [0.8450860381126404, 0.7037792801856995, 0.9499450922012329, 0.8121302723884583, 0.9946763515472412, 0.816746711730957, 0.4662463963031769, 0.5649397373199463, 0.7867223620414734, 0.7129095196723938, 0.4336201846599579, 0.9994826316833496, 0.2787272334098816, 0.9999070167541504, 0.9908583760261536, 0.7424963116645813, 0.8620147109031677, 0.9999213218688965, 0.2581188678741455, 0.9978200197219849], "n_positions_probed": 1, "per_restart_best": [4.90839958190918]}
+{"step": 313, "discrete_loss": 5.389908790588379, "best_sample_loss": 4.90839958190918, "soft_loss": 3.980087995529175, "best_discrete": 4.90839958190918, "best_soft": 2.7653648853302, "best_argmax": 5.101364612579346, "best_sampling": 4.90839958190918, "relax_gap": 0.2615667258638905, "n_match": 7, "g_first_norm": 135.25003051757812, "vocab_size": 50257, "entropy": 0.6704160571098328, "entropy_per_token": [0.6610744595527649, 0.6859601736068726, 0.20528942346572876, 0.5390985012054443, 0.03916709125041962, 0.5812815427780151, 1.0951135158538818, 0.6920078992843628, 0.6391857266426086, 1.0349465608596802, 1.4158532619476318, 0.004637510981410742, 1.6177195310592651, 3.197135924892791e-07, 0.05421953648328781, 0.9291836619377136, 0.5418285131454468, 0.0007558665820397437, 2.6522982120513916, 0.018699219450354576], "max_p": 0.7568463683128357, "max_p_per_token": [0.8334197402000427, 0.6998181343078613, 0.9488758444786072, 0.8004565834999084, 0.9945492148399353, 0.8134996294975281, 0.4669186472892761, 0.5489053130149841, 0.7919662594795227, 0.7135520577430725, 0.3871038556098938, 0.9994677901268005, 0.29767468571662903, 1.0, 0.9913349747657776, 0.7301353812217712, 0.8618884682655334, 0.99992835521698, 0.2598000168800354, 0.997632622718811], "n_positions_probed": 1, "per_restart_best": [4.90839958190918]}
+{"step": 314, "discrete_loss": 5.389908790588379, "best_sample_loss": 5.699094772338867, "soft_loss": 3.940953016281128, "best_discrete": 4.90839958190918, "best_soft": 2.7653648853302, "best_argmax": 5.101364612579346, "best_sampling": 4.90839958190918, "relax_gap": 0.2688275127841409, "n_match": 7, "g_first_norm": 134.42066955566406, "vocab_size": 50257, "entropy": 0.6747623682022095, "entropy_per_token": [0.701439619064331, 0.6902129650115967, 0.2089739441871643, 0.5558922290802002, 0.039922989904880524, 0.5832377076148987, 1.094176173210144, 0.6944280862808228, 0.634672999382019, 1.031355857849121, 1.4198092222213745, 0.004781581461429596, 1.5798749923706055, 3.2628773283249757e-07, 0.06146921217441559, 0.9620421528816223, 0.5415101647377014, 0.0006966213113628328, 2.6705379486083984, 0.02021227963268757], "max_p": 0.7568266987800598, "max_p_per_token": [0.8195467591285706, 0.697967529296875, 0.9476287961006165, 0.7901042699813843, 0.9944315552711487, 0.8117179870605469, 0.47092047333717346, 0.5352925658226013, 0.7963994145393372, 0.7148438692092896, 0.4170330762863159, 0.9994502663612366, 0.3175743520259857, 1.0, 0.9906101822853088, 0.719466507434845, 0.8615157604217529, 0.9999344348907471, 0.25469323992729187, 0.997403085231781], "n_positions_probed": 1, "per_restart_best": [4.90839958190918]}
+{"step": 315, "discrete_loss": 5.389908790588379, "best_sample_loss": 5.942903518676758, "soft_loss": 3.8990161418914795, "best_discrete": 4.90839958190918, "best_soft": 2.7653648853302, "best_argmax": 5.101364612579346, "best_sampling": 4.90839958190918, "relax_gap": 0.27660814062386924, "n_match": 7, "g_first_norm": 133.89266967773438, "vocab_size": 50257, "entropy": 0.6804812550544739, "entropy_per_token": [0.7442509531974792, 0.6953532099723816, 0.2125856876373291, 0.5734845399856567, 0.04073961824178696, 0.5856167078018188, 1.0932965278625488, 0.6958185434341431, 0.630921483039856, 1.0301501750946045, 1.4468363523483276, 0.004927328322082758, 1.5368956327438354, 3.336994325309206e-07, 0.059110041707754135, 0.9977250695228577, 0.5431840419769287, 0.0006429087952710688, 2.6962857246398926, 0.02179909683763981], "max_p": 0.7543678283691406, "max_p_per_token": [0.804291844367981, 0.6952288746833801, 0.9463950395584106, 0.7787461280822754, 0.9943034648895264, 0.8097207546234131, 0.4703153669834137, 0.5248096585273743, 0.8001875877380371, 0.7150396704673767, 0.41500404477119446, 0.9994325041770935, 0.3365088403224945, 1.0, 0.9910626411437988, 0.7080941796302795, 0.8605733513832092, 0.999940037727356, 0.24054214358329773, 0.997159481048584], "n_positions_probed": 1, "per_restart_best": [4.90839958190918]}
+{"step": 316, "discrete_loss": 5.389908790588379, "best_sample_loss": 6.460055828094482, "soft_loss": 3.8637855052948, "best_discrete": 4.90839958190918, "best_soft": 2.7653648853302, "best_argmax": 5.101364612579346, "best_sampling": 4.90839958190918, "relax_gap": 0.2831445474473387, "n_match": 7, "g_first_norm": 135.1612548828125, "vocab_size": 50257, "entropy": 0.6801498532295227, "entropy_per_token": [0.793310821056366, 0.7008922100067139, 0.21642716228961945, 0.5907163619995117, 0.04170471429824829, 0.5880508422851562, 1.090973138809204, 0.696631133556366, 0.628207802772522, 1.0305432081222534, 1.4722980260849, 0.0050924248062074184, 1.4941754341125488, 3.4096697731911263e-07, 0.05685386061668396, 1.0327558517456055, 0.4300477206707001, 0.0005919496761634946, 2.7102129459381104, 0.02351076900959015], "max_p": 0.75511234998703, "max_p_per_token": [0.7863197326660156, 0.6921569108963013, 0.9450716376304626, 0.7672921419143677, 0.994149923324585, 0.8077274560928345, 0.46991318464279175, 0.5165292024612427, 0.8032243847846985, 0.7142581939697266, 0.4185635447502136, 0.9994122982025146, 0.35595759749412537, 1.0, 0.9914900660514832, 0.6963261961936951, 0.9098469614982605, 0.9999452829360962, 0.23716861009597778, 0.9968929290771484], "n_positions_probed": 1, "per_restart_best": [4.90839958190918]}
+{"step": 317, "discrete_loss": 5.389908790588379, "best_sample_loss": 5.817320823669434, "soft_loss": 3.834186553955078, "best_discrete": 4.90839958190918, "best_soft": 2.7653648853302, "best_argmax": 5.101364612579346, "best_sampling": 4.90839958190918, "relax_gap": 0.2886360970244681, "n_match": 7, "g_first_norm": 133.55740356445312, "vocab_size": 50257, "entropy": 0.6875900626182556, "entropy_per_token": [0.8459351062774658, 0.706851065158844, 0.22026479244232178, 0.6078478097915649, 0.04265427216887474, 0.5901750922203064, 1.0877549648284912, 0.6970566511154175, 0.6265550851821899, 1.0335664749145508, 1.5123717784881592, 0.005270491819828749, 1.454463005065918, 3.4873932008849806e-07, 0.05460764467716217, 1.0684312582015991, 0.4393886625766754, 0.0005456172511912882, 2.7326717376708984, 0.025389274582266808], "max_p": 0.7518010139465332, "max_p_per_token": [0.7662398219108582, 0.6888152360916138, 0.943737804889679, 0.7553997039794922, 0.9939979314804077, 0.8058891892433167, 0.46977731585502625, 0.5111238956451416, 0.8054928779602051, 0.7122917175292969, 0.4032716751098633, 0.9993904829025269, 0.37513697147369385, 1.0, 0.9919093251228333, 0.6839883327484131, 0.9072193503379822, 0.9999499320983887, 0.22579310834407806, 0.99659663438797], "n_positions_probed": 1, "per_restart_best": [4.90839958190918]}
+{"step": 318, "discrete_loss": 5.389908790588379, "best_sample_loss": 4.946765422821045, "soft_loss": 3.8024299144744873, "best_discrete": 4.90839958190918, "best_soft": 2.7653648853302, "best_argmax": 5.101364612579346, "best_sampling": 4.90839958190918, "relax_gap": 0.2945279665744766, "n_match": 7, "g_first_norm": 140.44859313964844, "vocab_size": 50257, "entropy": 0.6892961263656616, "entropy_per_token": [0.9060187935829163, 0.7130374908447266, 0.22461465001106262, 0.6234867572784424, 0.04369215667247772, 0.5916891098022461, 1.0827064514160156, 0.6973147392272949, 0.6250892877578735, 1.035475492477417, 1.5288078784942627, 0.005465061403810978, 1.4161264896392822, 3.5605253856374475e-07, 0.05255364626646042, 1.1041712760925293, 0.4498330354690552, 0.0005007055588066578, 2.657918691635132, 0.02742018923163414], "max_p": 0.7515080571174622, "max_p_per_token": [0.742691159248352, 0.6854153871536255, 0.9422138333320618, 0.7443934679031372, 0.993830144405365, 0.8044266104698181, 0.4716120660305023, 0.5072266459465027, 0.80748450756073, 0.7103662490844727, 0.4218149483203888, 0.9993667006492615, 0.39462223649024963, 1.0, 0.9922886490821838, 0.6715155839920044, 0.9042477011680603, 0.9999544620513916, 0.24041830003261566, 0.996272087097168], "n_positions_probed": 1, "per_restart_best": [4.90839958190918]}
+{"step": 319, "discrete_loss": 5.389908790588379, "best_sample_loss": 5.870549201965332, "soft_loss": 3.8160812854766846, "best_discrete": 4.90839958190918, "best_soft": 2.7653648853302, "best_argmax": 5.101364612579346, "best_sampling": 4.90839958190918, "relax_gap": 0.2919952018223096, "n_match": 7, "g_first_norm": 140.92227172851562, "vocab_size": 50257, "entropy": 0.6982381939888, "entropy_per_token": [0.956850528717041, 0.7211183309555054, 0.22848275303840637, 0.6402451395988464, 0.045193351805210114, 0.5968579053878784, 1.0796247720718384, 0.6975468993186951, 0.6267359256744385, 1.042739987373352, 1.600933313369751, 0.005673846695572138, 1.3776227235794067, 3.6414661508388235e-07, 0.0503678172826767, 1.1441901922225952, 0.45685452222824097, 0.0004598545201588422, 2.6721115112304688, 0.021154403686523438], "max_p": 0.7459501028060913, "max_p_per_token": [0.7218103408813477, 0.679493248462677, 0.9408398270606995, 0.732241690158844, 0.993584930896759, 0.800770103931427, 0.4686030149459839, 0.5034130215644836, 0.8076444268226624, 0.7063432335853577, 0.3631074130535126, 0.9993409514427185, 0.4158298075199127, 1.0, 0.9926859140396118, 0.6566343307495117, 0.9021087288856506, 0.9999585151672363, 0.23725447058677673, 0.9973379969596863], "n_positions_probed": 1, "per_restart_best": [4.90839958190918]}
+{"step": 320, "discrete_loss": 5.389908790588379, "best_sample_loss": 4.970241546630859, "soft_loss": 3.796222686767578, "best_discrete": 4.90839958190918, "best_soft": 2.7653648853302, "best_argmax": 5.101364612579346, "best_sampling": 4.90839958190918, "relax_gap": 0.2956796053030851, "n_match": 7, "g_first_norm": 162.1327362060547, "vocab_size": 50257, "entropy": 0.7017715573310852, "entropy_per_token": [1.0152502059936523, 0.7267235517501831, 0.2336307168006897, 0.6530462503433228, 0.046699803322553635, 0.5982452034950256, 1.0720980167388916, 0.6976897716522217, 0.625534176826477, 1.0440300703048706, 1.583573818206787, 0.005910185165703297, 1.3389363288879395, 3.698915520544688e-07, 0.048396334052085876, 1.1807212829589844, 0.4633120000362396, 0.00042179360752925277, 2.678339719772339, 0.02287086471915245], "max_p": 0.7484065294265747, "max_p_per_token": [0.7030705809593201, 0.6766242980957031, 0.9390001893043518, 0.7236203551292419, 0.9933359622955322, 0.7993548512458801, 0.4778175950050354, 0.5010498762130737, 0.8090577125549316, 0.703876256942749, 0.4256480038166046, 0.9993116855621338, 0.43687400221824646, 1.0, 0.9930397272109985, 0.6433819532394409, 0.8999775648117065, 0.999962329864502, 0.24604539573192596, 0.9970822930335999], "n_positions_probed": 1, "per_restart_best": [4.90839958190918]}
+{"step": 321, "discrete_loss": 5.287844181060791, "best_sample_loss": 4.7656707763671875, "soft_loss": 3.767669677734375, "best_discrete": 4.7656707763671875, "best_soft": 2.7653648853302, "best_argmax": 5.101364612579346, "best_sampling": 4.7656707763671875, "relax_gap": 0.2874847388225148, "n_match": 7, "g_first_norm": 155.00164794921875, "vocab_size": 50257, "entropy": 0.7158210873603821, "entropy_per_token": [1.067461609840393, 0.7354128360748291, 0.23697182536125183, 0.6712779402732849, 0.048176344484090805, 0.6028971672058105, 1.0686962604522705, 0.69785475730896, 0.6314652562141418, 1.056459903717041, 1.693231225013733, 0.006148553919047117, 1.3093761205673218, 3.7780912975904357e-07, 0.04614226892590523, 1.2217628955841064, 0.4718037247657776, 0.000389597233152017, 2.726311445236206, 0.02458098717033863], "max_p": 0.7371298670768738, "max_p_per_token": [0.6800789833068848, 0.6703699231147766, 0.9377886652946472, 0.7088475227355957, 0.9930932521820068, 0.7955678105354309, 0.47358235716819763, 0.5010911822319031, 0.8069312572479248, 0.6983457803726196, 0.29445597529411316, 0.9992823004722595, 0.45329129695892334, 1.0, 0.9934365749359131, 0.6271671056747437, 0.8974054455757141, 0.9999654293060303, 0.21507364511489868, 0.9968239068984985], "n_positions_probed": 1, "per_restart_best": [4.7656707763671875]}
+{"step": 322, "discrete_loss": 5.389908790588379, "best_sample_loss": 4.772881984710693, "soft_loss": 3.776576280593872, "best_discrete": 4.7656707763671875, "best_soft": 2.7653648853302, "best_argmax": 5.101364612579346, "best_sampling": 4.7656707763671875, "relax_gap": 0.29932464030033995, "n_match": 7, "g_first_norm": 152.7869415283203, "vocab_size": 50257, "entropy": 0.7249318361282349, "entropy_per_token": [1.1329550743103027, 0.7422983646392822, 0.3068291246891022, 0.6846114993095398, 0.05012376978993416, 0.6019256114959717, 1.0585403442382812, 0.6979645490646362, 0.6349791884422302, 1.065580129623413, 1.7032594680786133, 0.006422917824238539, 1.274146556854248, 3.817052345311822e-07, 0.044470589607954025, 1.2585996389389038, 0.4810730814933777, 0.00035222709993831813, 2.727869987487793, 0.026633890345692635], "max_p": 0.7368665933609009, "max_p_per_token": [0.6520508527755737, 0.6662921905517578, 0.923858642578125, 0.6995384693145752, 0.9927682876586914, 0.7954748868942261, 0.48629024624824524, 0.49992281198501587, 0.8057854175567627, 0.6916837096214294, 0.31785476207733154, 0.999248206615448, 0.4694984257221222, 1.0, 0.9937304258346558, 0.6136006712913513, 0.8944010734558105, 0.9999690055847168, 0.23885324597358704, 0.9965094923973083], "n_positions_probed": 1, "per_restart_best": [4.7656707763671875]}
+{"step": 323, "discrete_loss": 5.389908790588379, "best_sample_loss": 4.7798027992248535, "soft_loss": 3.712249755859375, "best_discrete": 4.7656707763671875, "best_soft": 2.7653648853302, "best_argmax": 5.101364612579346, "best_sampling": 4.7656707763671875, "relax_gap": 0.3112592624310189, "n_match": 7, "g_first_norm": 162.1632080078125, "vocab_size": 50257, "entropy": 0.7320817112922668, "entropy_per_token": [1.199409008026123, 0.7479463815689087, 0.3134433329105377, 0.6895384192466736, 0.05158381536602974, 0.5999884605407715, 1.0522897243499756, 0.6980406045913696, 0.6345863342285156, 1.0671148300170898, 1.6886096000671387, 0.0066907466389238834, 1.2436819076538086, 3.853805026210466e-07, 0.042568836361169815, 1.2923879623413086, 0.489374577999115, 0.0003205189132131636, 2.7951507568359375, 0.02890772372484207], "max_p": 0.7356269955635071, "max_p_per_token": [0.623386800289154, 0.6640036106109619, 0.9216090440750122, 0.6914860606193542, 0.9925215840339661, 0.7958483099937439, 0.4896833002567291, 0.501833438873291, 0.8064271807670593, 0.6884142756462097, 0.37383589148521423, 0.9992152452468872, 0.48363152146339417, 1.0, 0.9940574169158936, 0.6009320616722107, 0.8916578888893127, 0.9999721050262451, 0.19786769151687622, 0.9961561560630798], "n_positions_probed": 1, "per_restart_best": [4.7656707763671875]}
+{"step": 324, "discrete_loss": 5.389908790588379, "best_sample_loss": 4.775333881378174, "soft_loss": 3.628269910812378, "best_discrete": 4.7656707763671875, "best_soft": 2.7653648853302, "best_argmax": 5.101364612579346, "best_sampling": 4.7656707763671875, "relax_gap": 0.32684020235223593, "n_match": 7, "g_first_norm": 154.3282470703125, "vocab_size": 50257, "entropy": 0.7403379678726196, "entropy_per_token": [1.241789698600769, 0.7590228915214539, 0.31972622871398926, 0.7041837573051453, 0.05364478379487991, 0.601571798324585, 1.0415418148040771, 0.6981028914451599, 0.6354589462280273, 1.075505018234253, 1.7341079711914062, 0.006980334874242544, 1.2198941707611084, 3.892669155902695e-07, 0.040449775755405426, 1.3284502029418945, 0.5036803483963013, 0.00028882548213005066, 2.811286687850952, 0.031071821227669716], "max_p": 0.7312727570533752, "max_p_per_token": [0.6062660813331604, 0.6558876633644104, 0.9194300770759583, 0.6801769733428955, 0.9921854138374329, 0.7938591241836548, 0.48699039220809937, 0.5022578835487366, 0.806220531463623, 0.6816877722740173, 0.3418005704879761, 0.9991794228553772, 0.49077028036117554, 1.0, 0.9944155216217041, 0.586502194404602, 0.8872355222702026, 0.9999750852584839, 0.20479987561702728, 0.9958146214485168], "n_positions_probed": 1, "per_restart_best": [4.7656707763671875]}
+{"step": 325, "discrete_loss": 5.389908790588379, "best_sample_loss": 4.745208263397217, "soft_loss": 3.582810878753662, "best_discrete": 4.745208263397217, "best_soft": 2.7653648853302, "best_argmax": 5.101364612579346, "best_sampling": 4.745208263397217, "relax_gap": 0.3352743027841569, "n_match": 7, "g_first_norm": 174.54989624023438, "vocab_size": 50257, "entropy": 0.7634233832359314, "entropy_per_token": [1.283046007156372, 0.7693725824356079, 0.3272669315338135, 0.7172433733940125, 0.05535557493567467, 0.9462096691131592, 1.0288426876068115, 0.6981202960014343, 0.6320573091506958, 1.0808591842651367, 1.725333571434021, 0.007361802272498608, 1.1989490985870361, 3.9068592627700127e-07, 0.038294024765491486, 1.3579461574554443, 0.5175498723983765, 0.00025965896202251315, 2.8508310317993164, 0.03356783464550972], "max_p": 0.722505509853363, "max_p_per_token": [0.5912647247314453, 0.6492242813110352, 0.9168016910552979, 0.6706058382987976, 0.9918965697288513, 0.6598870158195496, 0.49041813611984253, 0.5032950043678284, 0.8079068064689636, 0.6755079030990601, 0.3639610707759857, 0.9991311430931091, 0.4965812563896179, 1.0, 0.9947733283042908, 0.5751911401748657, 0.8828392624855042, 0.9999778270721436, 0.18543009459972382, 0.9954155683517456], "n_positions_probed": 1, "per_restart_best": [4.745208263397217]}
+{"step": 326, "discrete_loss": 5.409017086029053, "best_sample_loss": 4.745208263397217, "soft_loss": 3.5679233074188232, "best_discrete": 4.745208263397217, "best_soft": 2.7653648853302, "best_argmax": 5.101364612579346, "best_sampling": 4.745208263397217, "relax_gap": 0.3403749238222208, "n_match": 8, "g_first_norm": 161.58155822753906, "vocab_size": 50257, "entropy": 0.7188846468925476, "entropy_per_token": [1.2803899049758911, 0.7785724997520447, 0.3328079581260681, 0.7328053712844849, 0.057277876883745193, 0.9284217357635498, 0.0038679104764014482, 0.6981545686721802, 0.6318011283874512, 1.0984796285629272, 1.8056426048278809, 0.007735828869044781, 1.1831437349319458, 3.9062254586497147e-07, 0.035953618586063385, 1.386040210723877, 0.5353381633758545, 0.00023204906028695405, 2.8449246883392334, 0.03610185533761978], "max_p": 0.7415064573287964, "max_p_per_token": [0.5961021184921265, 0.644996166229248, 0.9148101806640625, 0.6571980118751526, 0.9915754199028015, 0.6707552075386047, 0.9995865225791931, 0.5051895380020142, 0.8080236315727234, 0.665229856967926, 0.24444188177585602, 0.9990837574005127, 0.49915367364883423, 1.0, 0.995154619216919, 0.5640989542007446, 0.8773313164710999, 0.9999804496765137, 0.2024145871400833, 0.9950026869773865], "n_positions_probed": 1, "per_restart_best": [4.745208263397217]}
+{"step": 327, "discrete_loss": 5.409017086029053, "best_sample_loss": 4.745208263397217, "soft_loss": 3.620522975921631, "best_discrete": 4.745208263397217, "best_soft": 2.7653648853302, "best_argmax": 5.101364612579346, "best_sampling": 4.745208263397217, "relax_gap": 0.3306504826407227, "n_match": 8, "g_first_norm": 217.78359985351562, "vocab_size": 50257, "entropy": 0.728812038898468, "entropy_per_token": [1.3327056169509888, 0.7891886830329895, 0.3442673087120056, 0.7394703030586243, 0.060114890336990356, 0.9164834022521973, 0.003862414276227355, 0.7973694205284119, 0.623015284538269, 1.114622712135315, 1.7428131103515625, 0.008242327719926834, 1.169582724571228, 3.922580162907252e-07, 0.03408268839120865, 1.412447452545166, 0.5457676649093628, 0.00020834157476201653, 2.9030134677886963, 0.03898172080516815], "max_p": 0.7396218180656433, "max_p_per_token": [0.5777345299720764, 0.6351925134658813, 0.9106511473655701, 0.6571815013885498, 0.9910853505134583, 0.679448664188385, 0.9995868802070618, 0.48937922716140747, 0.8120014667510986, 0.651624321937561, 0.31402140855789185, 0.9990184307098389, 0.49945634603500366, 1.0, 0.9954531192779541, 0.5534668564796448, 0.8738789558410645, 0.9999825954437256, 0.1587446928024292, 0.9945277571678162], "n_positions_probed": 1, "per_restart_best": [4.745208263397217]}
+{"step": 328, "discrete_loss": 5.0645928382873535, "best_sample_loss": 4.622314453125, "soft_loss": 3.5023467540740967, "best_discrete": 4.622314453125, "best_soft": 2.7653648853302, "best_argmax": 5.0645928382873535, "best_sampling": 4.622314453125, "relax_gap": 0.30846429991429425, "n_match": 8, "g_first_norm": 158.15928649902344, "vocab_size": 50257, "entropy": 0.7284104228019714, "entropy_per_token": [1.2975139617919922, 0.7910264730453491, 0.35383811593055725, 0.749531090259552, 0.06346988677978516, 0.9047045111656189, 0.0038082061801105738, 0.7991886734962463, 0.6252002120018005, 1.1488690376281738, 1.7546722888946533, 0.008727406151592731, 1.158942461013794, 3.9374259586111293e-07, 0.03236442804336548, 1.4413249492645264, 0.5573875308036804, 0.00018769281450659037, 2.835671901702881, 0.04177895560860634], "max_p": 0.7397004961967468, "max_p_per_token": [0.6004475355148315, 0.6396660804748535, 0.907092273235321, 0.6524883508682251, 0.9905101656913757, 0.6875141859054565, 0.9995929598808289, 0.48887768387794495, 0.8130020499229431, 0.6283267140388489, 0.26656973361968994, 0.9989557266235352, 0.5003054738044739, 1.0, 0.9957250356674194, 0.5409670472145081, 0.8702046275138855, 0.9999845027923584, 0.21972014009952545, 0.9940588474273682], "n_positions_probed": 1, "per_restart_best": [4.622314453125]}
+{"step": 329, "discrete_loss": 6.030261516571045, "best_sample_loss": 5.158712863922119, "soft_loss": 3.500159740447998, "best_discrete": 4.622314453125, "best_soft": 2.7653648853302, "best_argmax": 5.0645928382873535, "best_sampling": 4.622314453125, "relax_gap": 0.41956750452204683, "n_match": 8, "g_first_norm": 172.95840454101562, "vocab_size": 50257, "entropy": 0.704584002494812, "entropy_per_token": [1.366834282875061, 0.7969630360603333, 0.36405032873153687, 0.7561212778091431, 0.06561072915792465, 0.888986349105835, 0.003788003930822015, 0.8008559942245483, 0.6211903095245361, 0.5212751030921936, 1.7020306587219238, 0.009406098164618015, 1.1510593891143799, 3.972036779487098e-07, 0.030329162254929543, 1.4619300365447998, 0.567386269569397, 0.00017251298413611948, 2.938319206237793, 0.045370157808065414], "max_p": 0.7462809085845947, "max_p_per_token": [0.5716110467910767, 0.6352112889289856, 0.9032740592956543, 0.6504687070846558, 0.9901289343833923, 0.6973996758460999, 0.9995949864387512, 0.48845523595809937, 0.8146082758903503, 0.8670316338539124, 0.29928144812583923, 0.998866081237793, 0.5025394558906555, 1.0, 0.9960404634475708, 0.5335389971733093, 0.8668952584266663, 0.9999858140945435, 0.11723915487527847, 0.9934473633766174], "n_positions_probed": 1, "per_restart_best": [4.622314453125]}
+{"step": 330, "discrete_loss": 5.4102983474731445, "best_sample_loss": 3.7113780975341797, "soft_loss": 3.7476699352264404, "best_discrete": 3.7113780975341797, "best_soft": 2.7653648853302, "best_argmax": 5.0645928382873535, "best_sampling": 3.7113780975341797, "relax_gap": 0.30730808274616267, "n_match": 8, "g_first_norm": 190.98570251464844, "vocab_size": 50257, "entropy": 0.6920272707939148, "entropy_per_token": [1.2795264720916748, 0.7801330089569092, 0.3716314435005188, 0.766842246055603, 0.07133974134922028, 0.874600887298584, 0.0037760611157864332, 0.802483081817627, 0.6031414270401001, 0.5709875226020813, 1.614685297012329, 0.010195893235504627, 1.153792142868042, 3.9000545370981854e-07, 0.029161088168621063, 1.4862468242645264, 0.5738136172294617, 0.00015589460963383317, 2.7994279861450195, 0.048604413866996765], "max_p": 0.7614284753799438, "max_p_per_token": [0.6201057434082031, 0.6616792678833008, 0.9002825617790222, 0.6489698886871338, 0.9891323447227478, 0.7059677839279175, 0.9995960593223572, 0.49773454666137695, 0.8225576877593994, 0.8491287231445312, 0.4208225607872009, 0.9987610578536987, 0.49661388993263245, 1.0, 0.9962214231491089, 0.5223726630210876, 0.8647544980049133, 0.9999872446060181, 0.24099726974964142, 0.9928840398788452], "n_positions_probed": 1, "per_restart_best": [3.7113780975341797]}
+{"step": 331, "discrete_loss": 5.527318477630615, "best_sample_loss": 3.8005738258361816, "soft_loss": 3.9035675525665283, "best_discrete": 3.7113780975341797, "best_soft": 2.7653648853302, "best_argmax": 5.0645928382873535, "best_sampling": 3.7113780975341797, "relax_gap": 0.29376829499430923, "n_match": 8, "g_first_norm": 132.9159698486328, "vocab_size": 50257, "entropy": 0.7106976509094238, "entropy_per_token": [1.3619718551635742, 0.7837649583816528, 0.37788328528404236, 0.7829525470733643, 0.07424110919237137, 0.8673169612884521, 0.003832879476249218, 0.8051289319992065, 0.6119062304496765, 0.6266209483146667, 1.6466119289398193, 0.010860568843781948, 1.1585299968719482, 3.86465217161458e-07, 0.028070781379938126, 1.4901245832443237, 0.5837341547012329, 0.00014583471056539565, 2.946706533432007, 0.053547319024801254], "max_p": 0.7500158548355103, "max_p_per_token": [0.5848630666732788, 0.6591400504112244, 0.8978452086448669, 0.6397536993026733, 0.9886257648468018, 0.7103947997093201, 0.9995889067649841, 0.4884313941001892, 0.8187095522880554, 0.8274490237236023, 0.38547250628471375, 0.9986775517463684, 0.492416650056839, 1.0, 0.9963890314102173, 0.5239120721817017, 0.8613877296447754, 0.9999881982803345, 0.13526010513305664, 0.9920108914375305], "n_positions_probed": 1, "per_restart_best": [3.7113780975341797]}
+{"step": 332, "discrete_loss": 5.4102983474731445, "best_sample_loss": 4.191697597503662, "soft_loss": 3.8275229930877686, "best_discrete": 3.7113780975341797, "best_soft": 2.7653648853302, "best_argmax": 5.0645928382873535, "best_sampling": 3.7113780975341797, "relax_gap": 0.2925486272165756, "n_match": 8, "g_first_norm": 151.89727783203125, "vocab_size": 50257, "entropy": 0.715370237827301, "entropy_per_token": [1.3845078945159912, 0.7904329299926758, 0.383677214384079, 0.7980283498764038, 0.0789099633693695, 0.8580724000930786, 0.0038752141408622265, 0.8076434135437012, 0.6138564348220825, 0.6824086904525757, 1.6688721179962158, 0.011743023060262203, 1.1606240272521973, 3.811758233496221e-07, 0.027426831424236298, 1.5166966915130615, 0.5925135612487793, 0.00013291936193127185, 2.8701558113098145, 0.05782705172896385], "max_p": 0.7489867210388184, "max_p_per_token": [0.5803083777427673, 0.6537827253341675, 0.8954670429229736, 0.6358229517936707, 0.9878103733062744, 0.7157011032104492, 0.9995836615562439, 0.48656177520751953, 0.8178322911262512, 0.8051906228065491, 0.3487803637981415, 0.9985597729682922, 0.4855346381664276, 1.0, 0.9964895844459534, 0.5127815008163452, 0.8581178188323975, 0.9999892711639404, 0.21018092334270477, 0.9912402033805847], "n_positions_probed": 1, "per_restart_best": [3.7113780975341797]}
+{"step": 333, "discrete_loss": 5.527318477630615, "best_sample_loss": 4.40601110458374, "soft_loss": 3.7868409156799316, "best_discrete": 3.7113780975341797, "best_soft": 2.7653648853302, "best_argmax": 5.0645928382873535, "best_sampling": 3.7113780975341797, "relax_gap": 0.31488642621091933, "n_match": 8, "g_first_norm": 141.7013702392578, "vocab_size": 50257, "entropy": 0.7312130331993103, "entropy_per_token": [1.4505172967910767, 0.7985946536064148, 0.39169278740882874, 0.8162182569503784, 0.0826011449098587, 0.8493732213973999, 0.003929033409804106, 0.8097615242004395, 0.6185109615325928, 0.7392511367797852, 1.687366008758545, 0.01245461031794548, 1.1585310697555542, 4.258513115473761e-07, 0.026246249675750732, 1.5467314720153809, 0.6025127172470093, 0.00012067994248354807, 2.9668002128601074, 0.06304626911878586], "max_p": 0.7387076616287231, "max_p_per_token": [0.5541176795959473, 0.6463453769683838, 0.8922910094261169, 0.628205418586731, 0.9871517419815063, 0.7207459807395935, 0.9995768666267395, 0.4912191331386566, 0.8157088756561279, 0.7804464101791382, 0.2948934733867645, 0.9984657764434814, 0.4825231432914734, 1.0, 0.9966668486595154, 0.5020899772644043, 0.85462486743927, 0.9999903440475464, 0.13880762457847595, 0.9902827739715576], "n_positions_probed": 1, "per_restart_best": [3.7113780975341797]}
+{"step": 334, "discrete_loss": 5.271002769470215, "best_sample_loss": 4.552140712738037, "soft_loss": 3.721482038497925, "best_discrete": 3.7113780975341797, "best_soft": 2.7653648853302, "best_argmax": 5.0645928382873535, "best_sampling": 3.7113780975341797, "relax_gap": 0.29397076775355807, "n_match": 8, "g_first_norm": 192.55604553222656, "vocab_size": 50257, "entropy": 0.734495997428894, "entropy_per_token": [1.4346420764923096, 0.7986458539962769, 0.4013752043247223, 0.8333160281181335, 0.08784449845552444, 0.8368574380874634, 0.003937087021768093, 0.810084879398346, 0.6085035800933838, 0.8012652397155762, 1.6688072681427002, 0.013645661994814873, 1.154242992401123, 4.180471364634286e-07, 0.025409642606973648, 1.6500335931777954, 0.6116867661476135, 0.00010893982835114002, 2.88173770904541, 0.06777438521385193], "max_p": 0.7383851408958435, "max_p_per_token": [0.5698829889297485, 0.6503962278366089, 0.8883386850357056, 0.6248874664306641, 0.9862127304077148, 0.7277571558952332, 0.9995754361152649, 0.4887712001800537, 0.8200681805610657, 0.752724289894104, 0.28342267870903015, 0.9983031749725342, 0.47932201623916626, 1.0, 0.9967923760414124, 0.450573593378067, 0.8512622714042664, 0.9999914169311523, 0.21002230048179626, 0.9893985986709595], "n_positions_probed": 1, "per_restart_best": [3.7113780975341797]}
+{"step": 335, "discrete_loss": 6.247125625610352, "best_sample_loss": 4.8047075271606445, "soft_loss": 3.5802083015441895, "best_discrete": 3.7113780975341797, "best_soft": 2.7653648853302, "best_argmax": 5.0645928382873535, "best_sampling": 3.7113780975341797, "relax_gap": 0.4269031045466772, "n_match": 7, "g_first_norm": 170.47137451171875, "vocab_size": 50257, "entropy": 0.6766979098320007, "entropy_per_token": [1.5076851844787598, 0.8071765899658203, 0.41230908036231995, 0.8534837365150452, 0.09169755131006241, 0.8244178295135498, 0.0039452421478927135, 0.8120505809783936, 0.601113498210907, 0.8651568293571472, 1.6254191398620605, 0.014818436466157436, 1.146528959274292, 4.128865498387313e-07, 0.023954864591360092, 0.2502575218677521, 0.626568078994751, 0.00010009820107370615, 2.9959263801574707, 0.0713474228978157], "max_p": 0.7565818428993225, "max_p_per_token": [0.5406768321990967, 0.6421172618865967, 0.8838908076286316, 0.616387128829956, 0.9855076670646667, 0.7346426248550415, 0.9995741248130798, 0.48851484060287476, 0.8231547474861145, 0.720720648765564, 0.31256407499313354, 0.9981410503387451, 0.4778389036655426, 1.0, 0.9970048069953918, 0.9558207392692566, 0.8461952209472656, 0.9999921321868896, 0.1201772689819336, 0.9887162446975708], "n_positions_probed": 1, "per_restart_best": [3.7113780975341797]}
+{"step": 336, "discrete_loss": 6.24910306930542, "best_sample_loss": 4.682597637176514, "soft_loss": 4.422849655151367, "best_discrete": 3.7113780975341797, "best_soft": 2.7653648853302, "best_argmax": 5.0645928382873535, "best_sampling": 3.7113780975341797, "relax_gap": 0.29224248566555305, "n_match": 7, "g_first_norm": 156.9557647705078, "vocab_size": 50257, "entropy": 0.6721539497375488, "entropy_per_token": [1.501418113708496, 0.810070276260376, 0.41968676447868347, 0.8756846785545349, 0.10016048699617386, 0.8097922801971436, 0.003926730249077082, 0.8163936138153076, 0.5902329683303833, 0.9437843561172485, 1.6303017139434814, 0.015905208885669708, 1.143370270729065, 4.234985908624367e-07, 0.02258235216140747, 0.3037479519844055, 0.46218013763427734, 8.883012196747586e-05, 2.9184980392456055, 0.07525260746479034], "max_p": 0.7606567144393921, "max_p_per_token": [0.5500932931900024, 0.643239438533783, 0.8809962868690491, 0.6082325577735901, 0.983953058719635, 0.7424970865249634, 0.999575674533844, 0.4941956698894501, 0.8277470469474792, 0.6806877255439758, 0.30221468210220337, 0.9979876279830933, 0.4745715260505676, 1.0, 0.9972021579742432, 0.9418458938598633, 0.9028100967407227, 0.999993085861206, 0.19731758534908295, 0.9879742860794067], "n_positions_probed": 1, "per_restart_best": [3.7113780975341797]}
+{"step": 337, "discrete_loss": 6.24910306930542, "best_sample_loss": 4.961575984954834, "soft_loss": 4.370520114898682, "best_discrete": 3.7113780975341797, "best_soft": 2.7653648853302, "best_argmax": 5.0645928382873535, "best_sampling": 3.7113780975341797, "relax_gap": 0.3006164138392969, "n_match": 7, "g_first_norm": 138.26849365234375, "vocab_size": 50257, "entropy": 0.6888716816902161, "entropy_per_token": [1.5570709705352783, 0.8214592337608337, 0.4273841381072998, 0.8990195989608765, 0.10651655495166779, 0.7963500022888184, 0.003923638723790646, 0.8198813796043396, 0.5843520164489746, 1.0142486095428467, 1.6296963691711426, 0.01684071682393551, 1.1384940147399902, 4.341423505138664e-07, 0.02092229202389717, 0.37159162759780884, 0.4795405864715576, 8.017434447538108e-05, 3.011183261871338, 0.07887672632932663], "max_p": 0.7513962388038635, "max_p_per_token": [0.528193473815918, 0.6340397000312805, 0.8780683875083923, 0.5962540507316589, 0.982767641544342, 0.7495407462120056, 0.9995753169059753, 0.4908905029296875, 0.8300060629844666, 0.6408438682556152, 0.3093646466732025, 0.9978546500205994, 0.4723891317844391, 1.0, 0.9974368810653687, 0.921233057975769, 0.8973120450973511, 0.9999938011169434, 0.11488353461027145, 0.9872768521308899], "n_positions_probed": 1, "per_restart_best": [3.7113780975341797]}
+{"step": 338, "discrete_loss": 6.913060188293457, "best_sample_loss": 3.752370595932007, "soft_loss": 4.329150199890137, "best_discrete": 3.7113780975341797, "best_soft": 2.7653648853302, "best_argmax": 5.0645928382873535, "best_sampling": 3.7113780975341797, "relax_gap": 0.37377223950384536, "n_match": 7, "g_first_norm": 167.45310974121094, "vocab_size": 50257, "entropy": 0.6625904440879822, "entropy_per_token": [1.4871995449066162, 0.8143697381019592, 0.4356652796268463, 0.9227636456489563, 0.11665612459182739, 0.783928632736206, 0.003884886857122183, 0.8228987455368042, 0.570095419883728, 1.0924360752105713, 1.6386990547180176, 0.01829592138528824, 1.1291708946228027, 4.4172148250254395e-07, 0.01968865841627121, 0.46064138412475586, 0.49359411001205444, 7.038727926556021e-05, 2.3587794303894043, 0.0829700455069542], "max_p": 0.7640869617462158, "max_p_per_token": [0.566227912902832, 0.6494445204734802, 0.8748135566711426, 0.5879217982292175, 0.980847954750061, 0.7558805346488953, 0.9995792508125305, 0.4946931004524231, 0.8359581828117371, 0.5937193036079407, 0.294070839881897, 0.9976438879966736, 0.4758249819278717, 1.0, 0.9976099729537964, 0.8896788954734802, 0.8925902843475342, 0.9999946355819702, 0.4087560772895813, 0.9864841103553772], "n_positions_probed": 1, "per_restart_best": [3.7113780975341797]}
+{"step": 339, "discrete_loss": 6.334168910980225, "best_sample_loss": 4.797421932220459, "soft_loss": 5.265714168548584, "best_discrete": 3.7113780975341797, "best_soft": 2.7653648853302, "best_argmax": 5.0645928382873535, "best_sampling": 3.7113780975341797, "relax_gap": 0.16868112572424204, "n_match": 7, "g_first_norm": 166.28192138671875, "vocab_size": 50257, "entropy": 0.6807656288146973, "entropy_per_token": [1.4767029285430908, 0.8218143582344055, 0.4492631256580353, 0.9473730325698853, 0.12194551527500153, 0.7604304552078247, 0.003849421627819538, 0.8152246475219727, 0.5630785822868347, 1.1685774326324463, 1.5747804641723633, 0.019272619858384132, 1.1240344047546387, 4.504974526753358e-07, 0.019060298800468445, 0.5520316362380981, 0.5079611539840698, 6.606059469049796e-05, 2.5962753295898438, 0.09357114136219025], "max_p": 0.7553356289863586, "max_p_per_token": [0.5770648717880249, 0.6431654691696167, 0.8692265152931213, 0.5748119354248047, 0.9798347353935242, 0.767433762550354, 0.9995830655097961, 0.5153290629386902, 0.8386679291725159, 0.533433198928833, 0.3838214874267578, 0.9975036978721619, 0.4786472022533417, 1.0, 0.9976972937583923, 0.8507803082466125, 0.8882293701171875, 0.9999949932098389, 0.2270262986421585, 0.98446124792099], "n_positions_probed": 1, "per_restart_best": [3.7113780975341797]}
+{"step": 340, "discrete_loss": 6.247125625610352, "best_sample_loss": 3.7465076446533203, "soft_loss": 4.62762975692749, "best_discrete": 3.7113780975341797, "best_soft": 2.7653648853302, "best_argmax": 5.0645928382873535, "best_sampling": 3.7113780975341797, "relax_gap": 0.25923856277896357, "n_match": 7, "g_first_norm": 185.1236114501953, "vocab_size": 50257, "entropy": 0.6995616555213928, "entropy_per_token": [1.3649967908859253, 0.8381589651107788, 0.46136540174484253, 0.9790477752685547, 0.1252819448709488, 0.7564643621444702, 0.003734296653419733, 0.8213261961936951, 0.5626678466796875, 1.135378360748291, 1.6686115264892578, 0.020583488047122955, 1.1126865148544312, 4.5732360831607366e-07, 0.017958471551537514, 0.6682257056236267, 0.5231330394744873, 6.131056579761207e-05, 2.8303866386413574, 0.10116421431303024], "max_p": 0.7470088005065918, "max_p_per_token": [0.6072310209274292, 0.6365905404090881, 0.8644458651542664, 0.5561656951904297, 0.979211151599884, 0.7694692015647888, 0.9995961785316467, 0.5008286237716675, 0.8384014964103699, 0.583819568157196, 0.28544965386390686, 0.9973106384277344, 0.4908275604248047, 1.0, 0.997849702835083, 0.7881802320480347, 0.8835075497627258, 0.9999953508377075, 0.17837943136692047, 0.9829166531562805], "n_positions_probed": 1, "per_restart_best": [3.7113780975341797]}
+{"step": 341, "discrete_loss": 6.247125625610352, "best_sample_loss": 3.9222512245178223, "soft_loss": 4.180667877197266, "best_discrete": 3.7113780975341797, "best_soft": 2.7653648853302, "best_argmax": 5.0645928382873535, "best_sampling": 3.7113780975341797, "relax_gap": 0.33078536790448976, "n_match": 7, "g_first_norm": 162.42428588867188, "vocab_size": 50257, "entropy": 0.7178805470466614, "entropy_per_token": [1.3840168714523315, 0.8563841581344604, 0.46864598989486694, 1.001371145248413, 0.13350626826286316, 0.7473361492156982, 0.003735118778422475, 0.8253730535507202, 0.5571075677871704, 1.203992486000061, 1.6669728755950928, 0.02181445248425007, 1.1051514148712158, 4.65205431510185e-07, 0.01688234694302082, 0.8018132448196411, 0.5306293964385986, 5.651023093378171e-05, 2.9249391555786133, 0.1078806221485138], "max_p": 0.735815167427063, "max_p_per_token": [0.6040741801261902, 0.6235052943229675, 0.8615224361419678, 0.5472490191459656, 0.9775959253311157, 0.773705244064331, 0.9995954632759094, 0.4928549528121948, 0.8402981162071228, 0.5305219292640686, 0.2763294279575348, 0.9971264004707336, 0.49140673875808716, 1.0, 0.9979971051216125, 0.682285487651825, 0.8804752230644226, 0.9999958276748657, 0.15825465321540833, 0.9815101623535156], "n_positions_probed": 1, "per_restart_best": [3.7113780975341797]}
+{"step": 342, "discrete_loss": 6.247125625610352, "best_sample_loss": 3.6555962562561035, "soft_loss": 3.9495644569396973, "best_discrete": 3.6555962562561035, "best_soft": 2.7653648853302, "best_argmax": 5.0645928382873535, "best_sampling": 3.6555962562561035, "relax_gap": 0.367778928480597, "n_match": 7, "g_first_norm": 163.85787963867188, "vocab_size": 50257, "entropy": 0.7290424704551697, "entropy_per_token": [1.3880054950714111, 0.8679851293563843, 0.4775402843952179, 1.0280330181121826, 0.14245900511741638, 0.7411072254180908, 0.003712640842422843, 0.8293827772140503, 0.5503895282745361, 1.22550368309021, 1.6665451526641846, 0.023186640813946724, 1.0964996814727783, 4.715078603112488e-07, 0.016087673604488373, 0.8967920541763306, 0.5392965078353882, 5.1795621402561665e-05, 2.9728076457977295, 0.11546171456575394], "max_p": 0.7255662083625793, "max_p_per_token": [0.6079002022743225, 0.6141499876976013, 0.8580169677734375, 0.5352104902267456, 0.9758054614067078, 0.7764971852302551, 0.9995976090431213, 0.4849858283996582, 0.8426905870437622, 0.5195180177688599, 0.2791443169116974, 0.9969183206558228, 0.49476686120033264, 1.0, 0.9981058835983276, 0.506841242313385, 0.876717746257782, 0.9999961853027344, 0.16456648707389832, 0.9798949360847473], "n_positions_probed": 1, "per_restart_best": [3.6555962562561035]}
+{"step": 343, "discrete_loss": 5.1566033363342285, "best_sample_loss": 3.638512372970581, "soft_loss": 3.6051113605499268, "best_discrete": 3.638512372970581, "best_soft": 2.7653648853302, "best_argmax": 5.0645928382873535, "best_sampling": 3.638512372970581, "relax_gap": 0.30087479578897375, "n_match": 8, "g_first_norm": 167.67135620117188, "vocab_size": 50257, "entropy": 0.7330851554870605, "entropy_per_token": [1.3792449235916138, 0.8745867013931274, 0.4861101508140564, 1.0534389019012451, 0.15160244703292847, 0.7360134124755859, 0.00368863414041698, 0.8329707384109497, 0.545336127281189, 1.2514255046844482, 1.662402868270874, 0.024954132735729218, 1.0925389528274536, 4.702112050836149e-07, 0.015335145406425, 0.8610368967056274, 0.5478112697601318, 4.784078919328749e-05, 3.017306327819824, 0.125851571559906], "max_p": 0.7305256724357605, "max_p_per_token": [0.6167870759963989, 0.6102186441421509, 0.8542380928993225, 0.5228164196014404, 0.9739257097244263, 0.778484046459198, 0.9995998740196228, 0.4804912805557251, 0.8444932103157043, 0.5052651166915894, 0.28797006607055664, 0.9966464638710022, 0.4995945394039154, 1.0, 0.9982085227966309, 0.6346296072006226, 0.8725595474243164, 0.9999964237213135, 0.15694940090179443, 0.97763991355896], "n_positions_probed": 1, "per_restart_best": [3.638512372970581]}
+{"step": 344, "discrete_loss": 5.1566033363342285, "best_sample_loss": 3.736715078353882, "soft_loss": 3.3966097831726074, "best_discrete": 3.638512372970581, "best_soft": 2.7653648853302, "best_argmax": 5.0645928382873535, "best_sampling": 3.638512372970581, "relax_gap": 0.3413086945742817, "n_match": 8, "g_first_norm": 164.2200469970703, "vocab_size": 50257, "entropy": 0.7401660680770874, "entropy_per_token": [1.370861291885376, 0.8831706047058105, 0.49683934450149536, 1.0726447105407715, 0.2309037148952484, 0.7333166599273682, 0.0037064009811729193, 0.8365778923034668, 0.5391033291816711, 1.2789027690887451, 1.6521782875061035, 0.02659723535180092, 1.0934948921203613, 4.7077733711375913e-07, 0.014809089712798595, 0.8475576639175415, 0.5551011562347412, 4.517916022450663e-05, 3.030733585357666, 0.13677571713924408], "max_p": 0.7318490147590637, "max_p_per_token": [0.6245279312133789, 0.6010890603065491, 0.8491871953010559, 0.5146087408065796, 0.9585331082344055, 0.7792348861694336, 0.9995973706245422, 0.48765993118286133, 0.8468119502067566, 0.4920983612537384, 0.3073734641075134, 0.9963905215263367, 0.49165552854537964, 1.0, 0.9982808828353882, 0.6721153855323792, 0.8685227632522583, 0.9999966621398926, 0.1740763932466507, 0.9752198457717896], "n_positions_probed": 1, "per_restart_best": [3.638512372970581]}
+{"step": 345, "discrete_loss": 5.186835289001465, "best_sample_loss": 3.6887624263763428, "soft_loss": 3.340050458908081, "best_discrete": 3.638512372970581, "best_soft": 2.7653648853302, "best_argmax": 5.0645928382873535, "best_sampling": 3.638512372970581, "relax_gap": 0.35605233773461786, "n_match": 8, "g_first_norm": 157.1560516357422, "vocab_size": 50257, "entropy": 0.7550088167190552, "entropy_per_token": [1.3779895305633545, 0.8933596611022949, 0.5085830092430115, 1.091821312904358, 0.24333730340003967, 0.894466757774353, 0.0037437949795275927, 0.8399584889411926, 0.535608172416687, 1.3075828552246094, 1.6398823261260986, 0.028580304235219955, 1.0931380987167358, 4.6842455958540086e-07, 0.014224899001419544, 0.8349583745002747, 0.5633525848388672, 4.181627809884958e-05, 3.0814952850341797, 0.14805102348327637], "max_p": 0.7169502973556519, "max_p_per_token": [0.6254420876502991, 0.5888519287109375, 0.8435817956924438, 0.5039816498756409, 0.9557831883430481, 0.5137644410133362, 0.9995923638343811, 0.4876062273979187, 0.8480042219161987, 0.4804655611515045, 0.32564693689346313, 0.9960771203041077, 0.48545950651168823, 1.0, 0.998359739780426, 0.6982795596122742, 0.8640191555023193, 0.9999969005584717, 0.15142692625522614, 0.9726664423942566], "n_positions_probed": 1, "per_restart_best": [3.638512372970581]}
+{"step": 346, "discrete_loss": 5.186835289001465, "best_sample_loss": 3.6319618225097656, "soft_loss": 3.3377418518066406, "best_discrete": 3.6319618225097656, "best_soft": 2.7653648853302, "best_argmax": 5.0645928382873535, "best_sampling": 3.6319618225097656, "relax_gap": 0.3564974274613604, "n_match": 7, "g_first_norm": 169.15675354003906, "vocab_size": 50257, "entropy": 0.75359708070755, "entropy_per_token": [1.2944955825805664, 0.8930544853210449, 0.5243889093399048, 1.1065937280654907, 0.2559349834918976, 0.8916467428207397, 0.003837579395622015, 0.8433549404144287, 0.5303165912628174, 1.3434354066848755, 1.6272616386413574, 0.03130252659320831, 1.0882396697998047, 4.6414101007030695e-07, 0.013694335706532001, 0.8325526118278503, 0.5731798410415649, 3.8153732020873576e-05, 3.0613198280334473, 0.1572932004928589], "max_p": 0.718404233455658, "max_p_per_token": [0.6607287526130676, 0.5876644253730774, 0.8357823491096497, 0.49602627754211426, 0.9529692530632019, 0.5025084614753723, 0.9995803236961365, 0.4795484244823456, 0.8501334190368652, 0.46041548252105713, 0.33581653237342834, 0.9956398010253906, 0.48373275995254517, 1.0, 0.9984306693077087, 0.7137479186058044, 0.85889732837677, 0.9999972581863403, 0.18592777848243713, 0.9705367088317871], "n_positions_probed": 1, "per_restart_best": [3.6319618225097656]}
+{"step": 347, "discrete_loss": 5.194947719573975, "best_sample_loss": 3.6319618225097656, "soft_loss": 3.318934679031372, "best_discrete": 3.6319618225097656, "best_soft": 2.7653648853302, "best_argmax": 5.0645928382873535, "best_sampling": 3.6319618225097656, "relax_gap": 0.36112260253823114, "n_match": 8, "g_first_norm": 164.23158264160156, "vocab_size": 50257, "entropy": 0.7283293604850769, "entropy_per_token": [1.436781644821167, 0.9094331860542297, 0.539582371711731, 1.1252599954605103, 0.2652587592601776, 0.8911367654800415, 0.0039228592067956924, 0.004214438144117594, 0.5346910357475281, 1.3744919300079346, 1.630110263824463, 0.033642448484897614, 1.0864894390106201, 4.63909429981868e-07, 0.013193344697356224, 0.8232396841049194, 0.5853806138038635, 3.546240259311162e-05, 3.1411943435668945, 0.16852781176567078], "max_p": 0.7343393564224243, "max_p_per_token": [0.6071382164955139, 0.5580256581306458, 0.8280365467071533, 0.479257732629776, 0.9508242607116699, 0.49127528071403503, 0.9995693564414978, 0.9995348453521729, 0.8480235934257507, 0.4515536427497864, 0.33875060081481934, 0.9952571988105774, 0.47815218567848206, 1.0, 0.9984970092773438, 0.7310128808021545, 0.8524630665779114, 0.9999974966049194, 0.11152427643537521, 0.9678921699523926], "n_positions_probed": 1, "per_restart_best": [3.6319618225097656]}
+{"step": 348, "discrete_loss": 5.194947719573975, "best_sample_loss": 3.556000232696533, "soft_loss": 3.2824864387512207, "best_discrete": 3.556000232696533, "best_soft": 2.7653648853302, "best_argmax": 5.0645928382873535, "best_sampling": 3.556000232696533, "relax_gap": 0.3681386962984856, "n_match": 8, "g_first_norm": 211.75979614257812, "vocab_size": 50257, "entropy": 0.7293221354484558, "entropy_per_token": [1.3901091814041138, 0.9005246162414551, 0.5507506728172302, 1.1416444778442383, 0.284393310546875, 0.88862144947052, 0.00410211319103837, 0.0038325637578964233, 0.5385559797286987, 1.5150810480117798, 1.5823829174041748, 0.03483083099126816, 1.081554651260376, 4.612757038557902e-07, 0.012995771132409573, 0.8533892035484314, 0.6019242405891418, 3.257960270275362e-05, 3.0210304260253906, 0.18068602681159973], "max_p": 0.7379925847053528, "max_p_per_token": [0.625866174697876, 0.5789124965667725, 0.8222721219062805, 0.4635780453681946, 0.9463830590248108, 0.4875237047672272, 0.9995465874671936, 0.9995819926261902, 0.8544053435325623, 0.3747730851173401, 0.376882940530777, 0.995062530040741, 0.4829792082309723, 1.0, 0.9985238909721375, 0.7223063707351685, 0.8449212312698364, 0.999997615814209, 0.22135087847709656, 0.9649844765663147], "n_positions_probed": 1, "per_restart_best": [3.556000232696533]}
+{"step": 349, "discrete_loss": 6.334423065185547, "best_sample_loss": 5.272873878479004, "soft_loss": 3.097461700439453, "best_discrete": 3.556000232696533, "best_soft": 2.7653648853302, "best_argmax": 5.0645928382873535, "best_sampling": 3.556000232696533, "relax_gap": 0.5110112367670342, "n_match": 8, "g_first_norm": 175.99574279785156, "vocab_size": 50257, "entropy": 0.7455276846885681, "entropy_per_token": [1.4590615034103394, 0.9085787534713745, 0.5633893609046936, 1.1454150676727295, 0.3022982180118561, 0.8875635862350464, 0.004290353506803513, 0.003524347674101591, 0.5026834011077881, 1.6109038591384888, 1.5448921918869019, 0.0354480966925621, 1.0760712623596191, 4.5608106802319526e-07, 0.012515464797616005, 0.8717174530029297, 0.6166526079177856, 3.0408553357119672e-05, 3.16713285446167, 0.19838358461856842], "max_p": 0.7301995754241943, "max_p_per_token": [0.5993891358375549, 0.5668612122535706, 0.8155402541160583, 0.4603993594646454, 0.9421113133430481, 0.4771556258201599, 0.9995226860046387, 0.9996199607849121, 0.8671368360519409, 0.37037956714630127, 0.4043269157409668, 0.9949594736099243, 0.4871266186237335, 1.0, 0.9985860586166382, 0.7227229475975037, 0.8362600207328796, 0.9999978542327881, 0.10128901898860931, 0.9606069922447205], "n_positions_probed": 1, "per_restart_best": [3.556000232696533]}
+{"step": 350, "discrete_loss": 5.139954090118408, "best_sample_loss": 3.556000232696533, "soft_loss": 3.1303279399871826, "best_discrete": 3.556000232696533, "best_soft": 2.7653648853302, "best_argmax": 5.0645928382873535, "best_sampling": 3.556000232696533, "relax_gap": 0.39098134241991456, "n_match": 9, "g_first_norm": 177.0659942626953, "vocab_size": 50257, "entropy": 0.6680896282196045, "entropy_per_token": [1.398264765739441, 0.9089657068252563, 0.5762982368469238, 1.1533132791519165, 0.3234190046787262, 0.8844695687294006, 0.004444323014467955, 0.00326383369974792, 0.48884427547454834, 1.657314419746399, 0.051417026668787, 0.03597015142440796, 1.0727367401123047, 4.565019651181501e-07, 0.012385683134198189, 0.9088562726974487, 0.6365771293640137, 2.7377696824260056e-05, 3.0322275161743164, 0.2129966765642166], "max_p": 0.7621469497680664, "max_p_per_token": [0.6242765784263611, 0.5680574774742126, 0.8084548711776733, 0.44999977946281433, 0.9370169043540955, 0.4833330810070038, 0.9995027780532837, 0.9996514320373535, 0.872140109539032, 0.3280404806137085, 0.9931746125221252, 0.9948728680610657, 0.48556819558143616, 1.0, 0.9986035227775574, 0.7117726802825928, 0.8262559175491333, 0.9999980926513672, 0.205326110124588, 0.9568928480148315], "n_positions_probed": 1, "per_restart_best": [3.556000232696533]}
+{"step": 351, "discrete_loss": 5.139954090118408, "best_sample_loss": 3.580115556716919, "soft_loss": 3.3964684009552, "best_discrete": 3.556000232696533, "best_soft": 2.7653648853302, "best_argmax": 5.0645928382873535, "best_sampling": 3.556000232696533, "relax_gap": 0.3392025801388128, "n_match": 9, "g_first_norm": 144.80938720703125, "vocab_size": 50257, "entropy": 0.6884872317314148, "entropy_per_token": [1.5256235599517822, 0.9254251718521118, 0.5916317701339722, 1.1535645723342896, 0.339147686958313, 0.8841974139213562, 0.004654114134609699, 0.002859417349100113, 0.4919343590736389, 1.7071056365966797, 0.052466895431280136, 0.03643898293375969, 1.0786349773406982, 4.456819908682519e-07, 0.012178784236311913, 0.9382306337356567, 0.6629362106323242, 2.498948924767319e-05, 3.117387533187866, 0.24530164897441864], "max_p": 0.7511510252952576, "max_p_per_token": [0.5733815431594849, 0.5335073471069336, 0.7998522520065308, 0.4500748813152313, 0.9330175518989563, 0.4820787012577057, 0.9994761347770691, 0.999699592590332, 0.870359480381012, 0.28324589133262634, 0.9930154085159302, 0.9947903156280518, 0.49936339259147644, 1.0, 0.9986299276351929, 0.7103617787361145, 0.8120964765548706, 0.9999982118606567, 0.14168865978717804, 0.9483834505081177], "n_positions_probed": 1, "per_restart_best": [3.556000232696533]}
+{"step": 352, "discrete_loss": 4.594515323638916, "best_sample_loss": 3.556000232696533, "soft_loss": 3.359389066696167, "best_discrete": 3.556000232696533, "best_soft": 2.7653648853302, "best_argmax": 4.594515323638916, "best_sampling": 3.556000232696533, "relax_gap": 0.2688262351826292, "n_match": 10, "g_first_norm": 165.17848205566406, "vocab_size": 50257, "entropy": 0.6401290893554688, "entropy_per_token": [1.5449973344802856, 0.932195246219635, 0.6104413270950317, 1.1526432037353516, 0.361411452293396, 0.879977285861969, 0.005001131910830736, 0.0025695215445011854, 0.4943138360977173, 1.737589716911316, 0.053579043596982956, 0.03680095076560974, 0.11622343212366104, 4.425216104664287e-07, 0.012122733518481255, 0.9831845760345459, 0.6831533908843994, 2.239177774754353e-05, 2.9237608909606934, 0.27259424328804016], "max_p": 0.7747212648391724, "max_p_per_token": [0.565910816192627, 0.5045416951179504, 0.7888989448547363, 0.4512389600276947, 0.9273385405540466, 0.4771360456943512, 0.9994316697120667, 0.9997333884239197, 0.8689945936203003, 0.26457110047340393, 0.9928488731384277, 0.994723916053772, 0.9800814390182495, 1.0, 0.9986370205879211, 0.70020592212677, 0.8003532290458679, 0.9999984502792358, 0.2389167994260788, 0.940864086151123], "n_positions_probed": 1, "per_restart_best": [3.556000232696533]}
+{"step": 353, "discrete_loss": 7.097344875335693, "best_sample_loss": 4.651650905609131, "soft_loss": 3.473550796508789, "best_discrete": 3.556000232696533, "best_soft": 2.7653648853302, "best_argmax": 4.594515323638916, "best_sampling": 3.556000232696533, "relax_gap": 0.5105844710210033, "n_match": 10, "g_first_norm": 157.47756958007812, "vocab_size": 50257, "entropy": 0.6633355021476746, "entropy_per_token": [1.6626639366149902, 0.9319368600845337, 0.6381391286849976, 1.1502858400344849, 0.3775957524776459, 0.8806921243667603, 0.0051324861124157906, 0.0024363927077502012, 0.46014404296875, 1.8005032539367676, 0.0541459359228611, 0.036370255053043365, 0.12408307194709778, 4.279711731669522e-07, 0.012568369507789612, 1.0308599472045898, 0.6970281004905701, 2.1309673684299923e-05, 3.1181681156158447, 0.28393471240997314], "max_p": 0.764401376247406, "max_p_per_token": [0.5139026641845703, 0.4762810170650482, 0.7709904313087463, 0.4518311321735382, 0.9229878187179565, 0.4760962724685669, 0.9994149208068848, 0.9997490048408508, 0.8805983662605286, 0.27097731828689575, 0.9927650094032288, 0.9947841763496399, 0.9784180521965027, 1.0, 0.9985804557800293, 0.6893523931503296, 0.7912980318069458, 0.9999985694885254, 0.14232118427753448, 0.9376808404922485], "n_positions_probed": 1, "per_restart_best": [3.556000232696533]}
+{"step": 354, "discrete_loss": 4.5949835777282715, "best_sample_loss": 4.33553409576416, "soft_loss": 3.4598381519317627, "best_discrete": 3.556000232696533, "best_soft": 2.7653648853302, "best_argmax": 4.594515323638916, "best_sampling": 3.556000232696533, "relax_gap": 0.2470401485869329, "n_match": 10, "g_first_norm": 190.9368896484375, "vocab_size": 50257, "entropy": 0.6646329164505005, "entropy_per_token": [1.657392144203186, 0.9198099374771118, 0.6692137718200684, 1.1494567394256592, 0.4015897214412689, 0.8766102194786072, 0.005396916065365076, 0.002394504379481077, 0.4500506520271301, 1.8632540702819824, 0.05444325506687164, 0.03504125773906708, 0.12853378057479858, 4.2326132643211167e-07, 0.12974844872951508, 1.0948781967163086, 0.7129215002059937, 2.0355086235213093e-05, 2.850329875946045, 0.2915722727775574], "max_p": 0.7666410207748413, "max_p_per_token": [0.5166221261024475, 0.485312819480896, 0.7498266100883484, 0.45074892044067383, 0.916519820690155, 0.48139941692352295, 0.9993809461593628, 0.9997537732124329, 0.8837414979934692, 0.27497434616088867, 0.9927259683609009, 0.9949991703033447, 0.9774672389030457, 1.0, 0.9727824926376343, 0.670863926410675, 0.7806043028831482, 0.9999985694885254, 0.2496013194322586, 0.935496985912323], "n_positions_probed": 1, "per_restart_best": [3.556000232696533]}
+{"step": 355, "discrete_loss": 8.065155029296875, "best_sample_loss": 4.679350852966309, "soft_loss": 3.3073678016662598, "best_discrete": 3.556000232696533, "best_soft": 2.7653648853302, "best_argmax": 4.594515323638916, "best_sampling": 3.556000232696533, "relax_gap": 0.5899188807094018, "n_match": 9, "g_first_norm": 163.8981475830078, "vocab_size": 50257, "entropy": 0.6884158253669739, "entropy_per_token": [1.7327415943145752, 0.9172639846801758, 0.7038687467575073, 1.140945553779602, 0.421799898147583, 0.8778752088546753, 0.0055421944707632065, 0.0023109903559088707, 0.4089691936969757, 1.9026625156402588, 0.05408475548028946, 0.03501278907060623, 0.13450083136558533, 4.1100147996075975e-07, 0.13044509291648865, 1.2454066276550293, 0.7120169401168823, 1.9526165488059632e-05, 3.039652109146118, 0.30319690704345703], "max_p": 0.7525243759155273, "max_p_per_token": [0.4810042977333069, 0.47971996665000916, 0.7237473726272583, 0.46086978912353516, 0.9106935858726501, 0.4777107238769531, 0.9993619322776794, 0.9997634291648865, 0.8973928093910217, 0.2852640748023987, 0.9927834272384644, 0.9949901700019836, 0.976166307926178, 1.0, 0.9726139903068542, 0.5230709314346313, 0.7794016599655151, 0.9999986886978149, 0.1637669801712036, 0.9321666955947876], "n_positions_probed": 1, "per_restart_best": [3.556000232696533]}
+{"step": 356, "discrete_loss": 4.5949835777282715, "best_sample_loss": 5.305091381072998, "soft_loss": 3.3955588340759277, "best_discrete": 3.556000232696533, "best_soft": 2.7653648853302, "best_argmax": 4.594515323638916, "best_sampling": 3.556000232696533, "relax_gap": 0.261029168736514, "n_match": 10, "g_first_norm": 204.50082397460938, "vocab_size": 50257, "entropy": 0.7004154324531555, "entropy_per_token": [1.6965605020523071, 0.9009044170379639, 0.7256514430046082, 1.1274641752243042, 0.4527621269226074, 0.8717557787895203, 0.005885757971554995, 0.002344977343454957, 0.41067469120025635, 1.9677016735076904, 0.0533105731010437, 0.03461133688688278, 0.1364046335220337, 3.9989146216612426e-07, 0.13579529523849487, 1.2727868556976318, 1.0825116634368896, 1.843131576606538e-05, 2.820772409439087, 0.3103905916213989], "max_p": 0.7450485229492188, "max_p_per_token": [0.5045302510261536, 0.5228694081306458, 0.7065613865852356, 0.47322261333465576, 0.9016097784042358, 0.48155537247657776, 0.9993165731430054, 0.9997597336769104, 0.896520733833313, 0.2574297785758972, 0.9929051995277405, 0.9950483441352844, 0.9757310748100281, 1.0, 0.9711700081825256, 0.5251801609992981, 0.5164023637771606, 0.9999986886978149, 0.2511139512062073, 0.9300448894500732], "n_positions_probed": 1, "per_restart_best": [3.556000232696533]}
+{"step": 357, "discrete_loss": 7.097344875335693, "best_sample_loss": 4.480431079864502, "soft_loss": 3.240959882736206, "best_discrete": 3.556000232696533, "best_soft": 2.7653648853302, "best_argmax": 4.594515323638916, "best_sampling": 3.556000232696533, "relax_gap": 0.5433560099356292, "n_match": 10, "g_first_norm": 189.02996826171875, "vocab_size": 50257, "entropy": 0.7129718661308289, "entropy_per_token": [1.7383482456207275, 0.902243971824646, 0.7494469881057739, 1.117087960243225, 0.4823286831378937, 0.8690457940101624, 0.006114845629781485, 0.0023894784972071648, 0.3889179229736328, 2.033561944961548, 0.05172189697623253, 0.035550907254219055, 0.14371785521507263, 3.849610550332727e-07, 0.1321035474538803, 1.2945579290390015, 0.9674770832061768, 1.749910370563157e-05, 3.0191893577575684, 0.3256145715713501], "max_p": 0.7416189312934875, "max_p_per_token": [0.4850161373615265, 0.5174778699874878, 0.6855188012123108, 0.48671483993530273, 0.8924265503883362, 0.4794997274875641, 0.9992862343788147, 0.9997547268867493, 0.9034755825996399, 0.2202979177236557, 0.9931457042694092, 0.9948717951774597, 0.9741002917289734, 1.0, 0.9721784591674805, 0.5447292923927307, 0.6193880438804626, 0.9999988079071045, 0.13891619443893433, 0.9255812764167786], "n_positions_probed": 1, "per_restart_best": [3.556000232696533]}
+{"step": 358, "discrete_loss": 5.645564556121826, "best_sample_loss": 3.7318084239959717, "soft_loss": 3.0129153728485107, "best_discrete": 3.556000232696533, "best_soft": 2.7653648853302, "best_argmax": 4.594515323638916, "best_sampling": 3.556000232696533, "relax_gap": 0.46632168618434716, "n_match": 10, "g_first_norm": 196.44430541992188, "vocab_size": 50257, "entropy": 0.6598204970359802, "entropy_per_token": [1.7193995714187622, 0.8951452970504761, 0.7721547484397888, 1.1041182279586792, 0.5166964530944824, 0.8624041676521301, 0.0065527418628335, 0.0024453159421682358, 0.39139220118522644, 2.085659980773926, 0.04932834953069687, 0.03530322387814522, 0.1479116678237915, 3.7534513808168413e-07, 0.13322678208351135, 1.2957234382629395, 0.8958910703659058, 1.602185147930868e-05, 1.9484504461288452, 0.3345889151096344], "max_p": 0.7644404172897339, "max_p_per_token": [0.49193763732910156, 0.5035313963890076, 0.6631962060928345, 0.5009918808937073, 0.8814809322357178, 0.47779911756515503, 0.9992280006408691, 0.9997484087944031, 0.902373731136322, 0.21751920878887177, 0.9935067892074585, 0.9949020147323608, 0.9731426239013672, 1.0, 0.9718841314315796, 0.5646982192993164, 0.6737850904464722, 0.999998927116394, 0.5561808943748474, 0.9229021072387695], "n_positions_probed": 1, "per_restart_best": [3.556000232696533]}
+{"step": 359, "discrete_loss": 5.4236345291137695, "best_sample_loss": 4.819589138031006, "soft_loss": 4.1608052253723145, "best_discrete": 3.556000232696533, "best_soft": 2.7653648853302, "best_argmax": 4.594515323638916, "best_sampling": 3.556000232696533, "relax_gap": 0.23283820046551024, "n_match": 10, "g_first_norm": 203.79214477539062, "vocab_size": 50257, "entropy": 0.6776081919670105, "entropy_per_token": [1.739536166191101, 0.8688852190971375, 0.8008093237876892, 1.0940227508544922, 0.542290449142456, 0.8609851598739624, 0.006730262655764818, 0.0024277735501527786, 0.36139118671417236, 2.1611053943634033, 0.04823072999715805, 0.03597702085971832, 0.15920916199684143, 3.7939565800115815e-07, 0.13080891966819763, 1.3320285081863403, 0.8719627261161804, 1.5189569239737466e-05, 2.165814161300659, 0.3699331283569336], "max_p": 0.7598114013671875, "max_p_per_token": [0.477568119764328, 0.5683894157409668, 0.6300897002220154, 0.5133480429649353, 0.8733462691307068, 0.4937354028224945, 0.9992044568061829, 0.9997510313987732, 0.9121130704879761, 0.18306300044059753, 0.99366694688797, 0.9947648048400879, 0.9705836772918701, 1.0, 0.9725515246391296, 0.5569902658462524, 0.6897609829902649, 0.999998927116394, 0.45477399230003357, 0.9125280976295471], "n_positions_probed": 1, "per_restart_best": [3.556000232696533]}
+{"step": 360, "discrete_loss": 5.396121501922607, "best_sample_loss": 3.5899903774261475, "soft_loss": 3.8205933570861816, "best_discrete": 3.556000232696533, "best_soft": 2.7653648853302, "best_argmax": 4.594515323638916, "best_sampling": 3.556000232696533, "relax_gap": 0.2919741789125901, "n_match": 10, "g_first_norm": 212.0880584716797, "vocab_size": 50257, "entropy": 0.6980571150779724, "entropy_per_token": [1.6121641397476196, 0.843576967716217, 0.8269461989402771, 1.0832210779190063, 0.564700722694397, 0.856682300567627, 0.006980914622545242, 0.002414316637441516, 0.3420335054397583, 2.231576919555664, 0.04739343002438545, 0.03612758219242096, 0.16774943470954895, 3.778931159104104e-07, 0.12830641865730286, 1.3909502029418945, 0.855690598487854, 1.4363897207658738e-05, 2.5657882690429688, 0.3988233208656311], "max_p": 0.7456375956535339, "max_p_per_token": [0.421979159116745, 0.6021391153335571, 0.5963122844696045, 0.5241739749908447, 0.8657864332199097, 0.5078094005584717, 0.9991704225540161, 0.9997530579566956, 0.9182068109512329, 0.16628697514533997, 0.9937840700149536, 0.9947178959846497, 0.9685961008071899, 1.0, 0.9732425212860107, 0.5384870171546936, 0.7019349336624146, 0.9999990463256836, 0.23699557781219482, 0.9033762216567993], "n_positions_probed": 1, "per_restart_best": [3.556000232696533]}
+{"step": 361, "discrete_loss": 4.591933727264404, "best_sample_loss": 3.6089553833007812, "soft_loss": 3.106598377227783, "best_discrete": 3.556000232696533, "best_soft": 2.7653648853302, "best_argmax": 4.591933727264404, "best_sampling": 3.556000232696533, "relax_gap": 0.32346619926535697, "n_match": 10, "g_first_norm": 247.06820678710938, "vocab_size": 50257, "entropy": 0.7048296332359314, "entropy_per_token": [1.6442437171936035, 0.8579003810882568, 0.8461347222328186, 1.0665569305419922, 0.5857062339782715, 0.8464353680610657, 0.007366854697465897, 0.0024180973414331675, 0.33717894554138184, 2.278451681137085, 0.04586450010538101, 0.03466089814901352, 0.17307984828948975, 3.7007606579209096e-07, 0.12545056641101837, 1.4190860986709595, 0.8250336647033691, 1.3454493455355987e-05, 2.5825843811035156, 0.4184252619743347], "max_p": 0.7455148100852966, "max_p_per_token": [0.4041314721107483, 0.5938971042633057, 0.5769497752189636, 0.5399940609931946, 0.8577120900154114, 0.519838273525238, 0.9991176724433899, 0.999752938747406, 0.9193686246871948, 0.16013218462467194, 0.9940040707588196, 0.994956910610199, 0.9673126935958862, 1.0, 0.9740189909934998, 0.5365223288536072, 0.7224589586257935, 0.9999990463256836, 0.25310763716697693, 0.8970214128494263], "n_positions_probed": 1, "per_restart_best": [3.556000232696533]}
+{"step": 362, "discrete_loss": 4.591865062713623, "best_sample_loss": 3.5597996711730957, "soft_loss": 2.993680715560913, "best_discrete": 3.556000232696533, "best_soft": 2.7653648853302, "best_argmax": 4.591865062713623, "best_sampling": 3.556000232696533, "relax_gap": 0.34804688842669124, "n_match": 10, "g_first_norm": 174.33985900878906, "vocab_size": 50257, "entropy": 0.7194001078605652, "entropy_per_token": [1.662834644317627, 0.8919883966445923, 0.837221622467041, 1.074058175086975, 0.6059403419494629, 0.8437318801879883, 0.0074303774163126945, 0.002508261241018772, 0.33263593912124634, 2.3149828910827637, 0.04240816831588745, 0.03476390987634659, 0.18087860941886902, 3.4888634559138154e-07, 0.12462028861045837, 1.4594820737838745, 0.7987858653068542, 1.2847160178353079e-05, 2.7443618774414062, 0.42935431003570557], "max_p": 0.7396227717399597, "max_p_per_token": [0.3887402415275574, 0.5151911377906799, 0.5952650904655457, 0.5308579206466675, 0.8494971990585327, 0.5182073712348938, 0.9991084933280945, 0.9997430443763733, 0.9203650951385498, 0.17287960648536682, 0.9945104122161865, 0.9949212670326233, 0.9654500484466553, 1.0, 0.9742427468299866, 0.5272621512413025, 0.7399019598960876, 0.9999991655349731, 0.21290114521980286, 0.8934121131896973], "n_positions_probed": 1, "per_restart_best": [3.556000232696533]}
+{"step": 363, "discrete_loss": 4.591865062713623, "best_sample_loss": 3.5404751300811768, "soft_loss": 2.8098878860473633, "best_discrete": 3.5404751300811768, "best_soft": 2.7653648853302, "best_argmax": 4.591865062713623, "best_sampling": 3.5404751300811768, "relax_gap": 0.38807263548227544, "n_match": 10, "g_first_norm": 161.77052307128906, "vocab_size": 50257, "entropy": 0.7338599562644958, "entropy_per_token": [1.6816792488098145, 0.8954838514328003, 0.8533622026443481, 1.1100544929504395, 0.6367055177688599, 0.8387889862060547, 0.007657125126570463, 0.0025888546369969845, 0.32957276701927185, 2.34951114654541, 0.040697745978832245, 0.035221729427576065, 0.18778780102729797, 3.3078177352763305e-07, 0.12269265949726105, 1.4938559532165527, 0.7865996360778809, 1.2297065040911548e-05, 2.863783359527588, 0.4411422908306122], "max_p": 0.7323140501976013, "max_p_per_token": [0.3737255334854126, 0.49292585253715515, 0.5836403965950012, 0.526032567024231, 0.8366979956626892, 0.5225973129272461, 0.9990770816802979, 0.999734103679657, 0.9208198189735413, 0.15843996405601501, 0.9947575330734253, 0.9948257803916931, 0.9637724757194519, 1.0, 0.9747564792633057, 0.518049418926239, 0.7489463686943054, 0.9999991655349731, 0.147968590259552, 0.8895130753517151], "n_positions_probed": 1, "per_restart_best": [3.5404751300811768]}
+{"step": 364, "discrete_loss": 4.591865062713623, "best_sample_loss": 3.4890003204345703, "soft_loss": 2.743114948272705, "best_discrete": 3.4890003204345703, "best_soft": 2.743114948272705, "best_argmax": 4.591865062713623, "best_sampling": 3.4890003204345703, "relax_gap": 0.4026142077764748, "n_match": 10, "g_first_norm": 161.67921447753906, "vocab_size": 50257, "entropy": 0.7473721504211426, "entropy_per_token": [1.6701079607009888, 0.890280544757843, 0.8757143616676331, 1.102506399154663, 0.8768354654312134, 0.8334105014801025, 0.007918656803667545, 0.002705249935388565, 0.3295922875404358, 2.386845350265503, 0.03930831700563431, 0.03544260933995247, 0.1941888928413391, 3.1424889357367647e-07, 0.12274670600891113, 1.5131549835205078, 0.7955732345581055, 1.147280909208348e-05, 2.821133613586426, 0.4499664902687073], "max_p": 0.7226876616477966, "max_p_per_token": [0.36505863070487976, 0.4864003360271454, 0.5668088793754578, 0.5318915247917175, 0.6598081588745117, 0.5261132717132568, 0.9990407824516296, 0.999721109867096, 0.9203339219093323, 0.1509581357240677, 0.9949571490287781, 0.9947768449783325, 0.9621992707252502, 1.0, 0.9747520685195923, 0.5141342878341675, 0.746696949005127, 0.9999992847442627, 0.17352697253227234, 0.8865751028060913], "n_positions_probed": 1, "per_restart_best": [3.4890003204345703]}
+{"step": 365, "discrete_loss": 4.795076847076416, "best_sample_loss": 3.4549195766448975, "soft_loss": 2.7395687103271484, "best_discrete": 3.4549195766448975, "best_soft": 2.7395687103271484, "best_argmax": 4.591865062713623, "best_sampling": 3.4549195766448975, "relax_gap": 0.42867053069285466, "n_match": 10, "g_first_norm": 168.46560668945312, "vocab_size": 50257, "entropy": 0.7506446838378906, "entropy_per_token": [1.663745641708374, 0.8856715559959412, 0.8884440064430237, 1.0952847003936768, 0.820617139339447, 0.8265354633331299, 0.008144414983689785, 0.0028684716671705246, 0.326104074716568, 2.419299840927124, 0.037626445293426514, 0.036068353801965714, 0.2021198868751526, 2.9471976858985727e-07, 0.12145154178142548, 1.5186614990234375, 0.7963595390319824, 1.0880462468776386e-05, 2.9066872596740723, 0.45719197392463684], "max_p": 0.7224082946777344, "max_p_per_token": [0.3552679121494293, 0.47927770018577576, 0.5657783150672913, 0.5382351279258728, 0.7026945352554321, 0.5315914750099182, 0.9990091323852539, 0.9997026324272156, 0.9210318326950073, 0.14930380880832672, 0.9951980710029602, 0.994655966758728, 0.9602356553077698, 1.0, 0.975100576877594, 0.5147340297698975, 0.7484961152076721, 0.9999992847442627, 0.13369596004486084, 0.8841577172279358], "n_positions_probed": 1, "per_restart_best": [3.4549195766448975]}
+{"step": 366, "discrete_loss": 4.795076847076416, "best_sample_loss": 3.443474531173706, "soft_loss": 2.6987762451171875, "best_discrete": 3.443474531173706, "best_soft": 2.6987762451171875, "best_argmax": 4.591865062713623, "best_sampling": 3.443474531173706, "relax_gap": 0.4371776863674988, "n_match": 11, "g_first_norm": 180.26889038085938, "vocab_size": 50257, "entropy": 0.7445471882820129, "entropy_per_token": [1.6377434730529785, 0.8761553764343262, 0.9114438891410828, 1.0835260152816772, 0.8065260648727417, 0.8195207118988037, 3.7124154914636165e-05, 0.003029232146218419, 0.32407692074775696, 2.454984664916992, 0.03633598983287811, 0.03617830574512482, 0.20922377705574036, 2.791981614791439e-07, 0.12152281403541565, 1.510807991027832, 0.8084069490432739, 1.0128652320418041e-05, 2.788024425506592, 0.4633897840976715], "max_p": 0.7250273823738098, "max_p_per_token": [0.35103365778923035, 0.48171842098236084, 0.5500130653381348, 0.5483540892601013, 0.7163411974906921, 0.5366529226303101, 0.9999973773956299, 0.9996843338012695, 0.9213016629219055, 0.1433807611465454, 0.995383083820343, 0.9946315288543701, 0.9584558606147766, 1.0, 0.9750936627388, 0.5210460424423218, 0.7440646290779114, 0.9999992847442627, 0.181303471326828, 0.8820933699607849], "n_positions_probed": 1, "per_restart_best": [3.443474531173706]}
+{"step": 367, "discrete_loss": 4.716332912445068, "best_sample_loss": 3.717667818069458, "soft_loss": 2.7055563926696777, "best_discrete": 3.443474531173706, "best_soft": 2.6987762451171875, "best_argmax": 4.591865062713623, "best_sampling": 3.443474531173706, "relax_gap": 0.42634321136862924, "n_match": 11, "g_first_norm": 161.25950622558594, "vocab_size": 50257, "entropy": 0.7585768103599548, "entropy_per_token": [1.6730220317840576, 0.8739112615585327, 0.935488760471344, 1.0956175327301025, 0.8141528367996216, 0.815085768699646, 3.821559585048817e-05, 0.002480115508660674, 0.3219570219516754, 2.482663154602051, 0.03539188951253891, 0.03662268444895744, 0.2164180874824524, 2.6106016548510524e-07, 0.12069550156593323, 1.5239394903182983, 0.8071759939193726, 9.712101928016637e-06, 2.9493937492370605, 0.467471718788147], "max_p": 0.7222575545310974, "max_p_per_token": [0.3482714295387268, 0.5156852602958679, 0.5360453724861145, 0.5348560214042664, 0.7161901593208313, 0.5425267815589905, 0.9999973773956299, 0.9997350573539734, 0.9215282201766968, 0.13973942399024963, 0.9955152869224548, 0.9945400357246399, 0.9566428661346436, 1.0, 0.9753168821334839, 0.515006422996521, 0.7467931509017944, 0.9999994039535522, 0.12600389122962952, 0.880757749080658], "n_positions_probed": 1, "per_restart_best": [3.443474531173706]}
+{"step": 368, "discrete_loss": 4.716332912445068, "best_sample_loss": 3.4797863960266113, "soft_loss": 2.677603244781494, "best_discrete": 3.443474531173706, "best_soft": 2.677603244781494, "best_argmax": 4.591865062713623, "best_sampling": 3.443474531173706, "relax_gap": 0.43227009320820914, "n_match": 11, "g_first_norm": 206.3557891845703, "vocab_size": 50257, "entropy": 0.7457506060600281, "entropy_per_token": [1.6463708877563477, 0.8639705181121826, 0.9577301740646362, 1.0719208717346191, 0.802176833152771, 0.8075498938560486, 4.0091603295877576e-05, 0.002568951342254877, 0.31877636909484863, 2.517756223678589, 0.034324318170547485, 0.03647839277982712, 0.22263365983963013, 2.501773508356564e-07, 0.12101259082555771, 1.5077894926071167, 0.8204543590545654, 9.037215022544842e-06, 2.711233139038086, 0.47221532464027405], "max_p": 0.7273870706558228, "max_p_per_token": [0.36076974868774414, 0.5033556818962097, 0.523641049861908, 0.5564581751823425, 0.7258565425872803, 0.5470989942550659, 0.9999971389770508, 0.9997244477272034, 0.9222255349159241, 0.13025806844234467, 0.9956679344177246, 0.9945639371871948, 0.955050528049469, 1.0, 0.9752463698387146, 0.5232930183410645, 0.7413751482963562, 0.9999994039535522, 0.21394135057926178, 0.8792181611061096], "n_positions_probed": 1, "per_restart_best": [3.443474531173706]}
+{"step": 369, "discrete_loss": 4.716332912445068, "best_sample_loss": 5.172976970672607, "soft_loss": 2.7153403759002686, "best_discrete": 3.443474531173706, "best_soft": 2.677603244781494, "best_argmax": 4.591865062713623, "best_sampling": 3.443474531173706, "relax_gap": 0.42426872184207914, "n_match": 11, "g_first_norm": 159.525634765625, "vocab_size": 50257, "entropy": 0.7012529969215393, "entropy_per_token": [1.6941280364990234, 0.8614418506622314, 0.9804956316947937, 1.0849782228469849, 0.8134821653366089, 0.8030954599380493, 4.1225557652069256e-05, 0.0026560984551906586, 0.3159867227077484, 1.3190796375274658, 0.033570148050785065, 0.0367320217192173, 0.23012040555477142, 2.3400988879984652e-07, 0.11942262947559357, 1.5278546810150146, 0.8124883770942688, 8.858054570737295e-06, 2.9148786067962646, 0.4745987057685852], "max_p": 0.7523779273033142, "max_p_per_token": [0.350699245929718, 0.5383722186088562, 0.5104370713233948, 0.542687177658081, 0.7213215827941895, 0.5532733201980591, 0.9999971389770508, 0.9997140765190125, 0.9226945042610168, 0.7123124599456787, 0.9957712292671204, 0.9945066571235657, 0.9531221389770508, 1.0, 0.9756655097007751, 0.5109545588493347, 0.7471938729286194, 0.9999994039535522, 0.14034435153007507, 0.878491222858429], "n_positions_probed": 1, "per_restart_best": [3.443474531173706]}
+{"step": 370, "discrete_loss": 4.795076847076416, "best_sample_loss": 4.748392581939697, "soft_loss": 3.3090641498565674, "best_discrete": 3.443474531173706, "best_soft": 2.677603244781494, "best_argmax": 4.591865062713623, "best_sampling": 3.443474531173706, "relax_gap": 0.30990383358003504, "n_match": 11, "g_first_norm": 182.39739990234375, "vocab_size": 50257, "entropy": 0.7446207404136658, "entropy_per_token": [1.7006694078445435, 0.8612971305847168, 0.9691464900970459, 1.1007579565048218, 0.8536851406097412, 0.8022962808609009, 4.279031782061793e-05, 0.0026395616587251425, 0.3223420977592468, 2.0536041259765625, 0.0412258580327034, 0.03900735452771187, 0.2427171766757965, 2.2935574861548957e-07, 0.1215703934431076, 1.579421043395996, 0.8275110721588135, 8.134945346682798e-06, 2.8966732025146484, 0.47779813408851624], "max_p": 0.7337204813957214, "max_p_per_token": [0.34878775477409363, 0.4880693554878235, 0.5393314957618713, 0.5263959169387817, 0.6978799700737, 0.5410739183425903, 0.9999970197677612, 0.9997159838676453, 0.9202885627746582, 0.4287927448749542, 0.9947866201400757, 0.9940909743309021, 0.9498560428619385, 1.0, 0.9751179218292236, 0.48078423738479614, 0.7398614287376404, 0.9999995231628418, 0.17209604382514954, 0.8774836659431458], "n_positions_probed": 1, "per_restart_best": [3.443474531173706]}
+{"step": 371, "discrete_loss": 4.795076847076416, "best_sample_loss": 3.4182803630828857, "soft_loss": 2.8667984008789062, "best_discrete": 3.4182803630828857, "best_soft": 2.677603244781494, "best_argmax": 4.591865062713623, "best_sampling": 3.4182803630828857, "relax_gap": 0.40213713099784654, "n_match": 10, "g_first_norm": 147.5864715576172, "vocab_size": 50257, "entropy": 0.758191704750061, "entropy_per_token": [1.6918120384216309, 0.857765793800354, 0.9716012477874756, 1.08455491065979, 0.8489036560058594, 0.7972513437271118, 4.4104195694671944e-05, 0.0027432844508439302, 0.3232566714286804, 2.2954049110412598, 0.04060628265142441, 0.040839117020368576, 0.2538648247718811, 2.211850471667276e-07, 0.12029310315847397, 1.57496976852417, 0.8189864158630371, 7.7650292951148e-06, 2.9593918323516846, 0.48153701424598694], "max_p": 0.7250459790229797, "max_p_per_token": [0.34828925132751465, 0.49545446038246155, 0.5475442409515381, 0.5429242253303528, 0.7001913189888, 0.5346403121948242, 0.9999969005584717, 0.999703586101532, 0.919485330581665, 0.2757085859775543, 0.9948728680610657, 0.9937487840652466, 0.9468555450439453, 1.0, 0.9754543304443359, 0.47969797253608704, 0.7452057003974915, 0.9999995231628418, 0.12484368681907654, 0.8763021230697632], "n_positions_probed": 1, "per_restart_best": [3.4182803630828857]}
+{"step": 372, "discrete_loss": 4.716332912445068, "best_sample_loss": 3.346905469894409, "soft_loss": 2.695873260498047, "best_discrete": 3.346905469894409, "best_soft": 2.677603244781494, "best_argmax": 4.591865062713623, "best_sampling": 3.346905469894409, "relax_gap": 0.4283963175321233, "n_match": 9, "g_first_norm": 180.2739715576172, "vocab_size": 50257, "entropy": 0.7529705166816711, "entropy_per_token": [1.6573514938354492, 0.8454874753952026, 0.9822337627410889, 1.0582077503204346, 0.821736216545105, 0.7918548583984375, 4.5752007281407714e-05, 0.002905746456235647, 0.32376497983932495, 2.417726993560791, 0.03890516608953476, 0.0414748340845108, 0.2680458724498749, 2.130000211764127e-07, 0.12094487249851227, 1.5527997016906738, 0.8300125002861023, 7.253461717482423e-06, 2.821220874786377, 0.484683632850647], "max_p": 0.7278653383255005, "max_p_per_token": [0.35032644867897034, 0.5009312033653259, 0.5404941439628601, 0.5667409300804138, 0.7151370048522949, 0.5302388072013855, 0.9999967813491821, 0.9996838569641113, 0.9189070463180542, 0.22771234810352325, 0.9951140880584717, 0.9936287999153137, 0.9439336657524109, 1.0, 0.9752969145774841, 0.4898715913295746, 0.7407010197639465, 0.9999995231628418, 0.1932779848575592, 0.8753142356872559], "n_positions_probed": 1, "per_restart_best": [3.346905469894409]}
+{"step": 373, "discrete_loss": 4.716332912445068, "best_sample_loss": 4.097513198852539, "soft_loss": 2.6708810329437256, "best_discrete": 3.346905469894409, "best_soft": 2.6708810329437256, "best_argmax": 4.591865062713623, "best_sampling": 3.346905469894409, "relax_gap": 0.43369539798684986, "n_match": 9, "g_first_norm": 150.65118408203125, "vocab_size": 50257, "entropy": 0.7702662348747253, "entropy_per_token": [1.7037897109985352, 0.8433824181556702, 1.0039032697677612, 1.065846562385559, 0.8254182934761047, 0.7883213758468628, 4.726719271275215e-05, 0.0030701905488967896, 0.32230091094970703, 2.5045969486236572, 0.038102228194475174, 0.04261296987533569, 0.2742235064506531, 2.0230385189279332e-07, 0.11959076672792435, 1.5494779348373413, 0.8274552226066589, 7.106579687388148e-06, 3.007138729095459, 0.4860392212867737], "max_p": 0.7210511565208435, "max_p_per_token": [0.34288290143013, 0.5271239280700684, 0.522560179233551, 0.5590387582778931, 0.7105010151863098, 0.5296725034713745, 0.9999966621398926, 0.9996638298034668, 0.9189020395278931, 0.182486429810524, 0.9952238202095032, 0.9934074878692627, 0.9422231316566467, 1.0, 0.9756553769111633, 0.4890427589416504, 0.7434053421020508, 0.9999995231628418, 0.11426783353090286, 0.8749685883522034], "n_positions_probed": 1, "per_restart_best": [3.346905469894409]}
+{"step": 374, "discrete_loss": 5.736664772033691, "best_sample_loss": 4.39506721496582, "soft_loss": 2.6194920539855957, "best_discrete": 3.346905469894409, "best_soft": 2.6194920539855957, "best_argmax": 4.591865062713623, "best_sampling": 3.346905469894409, "relax_gap": 0.5433771785383642, "n_match": 8, "g_first_norm": 213.89865112304688, "vocab_size": 50257, "entropy": 0.7616065144538879, "entropy_per_token": [1.6675095558166504, 0.8343364000320435, 1.0086946487426758, 1.0419323444366455, 0.8053061962127686, 0.782807469367981, 4.9148988182423636e-05, 0.003227155888453126, 0.31907832622528076, 2.565537452697754, 0.036668069660663605, 0.04274214431643486, 0.27991020679473877, 1.9549041496702557e-07, 0.27210259437561035, 1.5173295736312866, 0.8432447910308838, 6.454185040638549e-06, 2.7236533164978027, 0.48799362778663635], "max_p": 0.7243852615356445, "max_p_per_token": [0.35763734579086304, 0.5046855807304382, 0.522333025932312, 0.5789637565612793, 0.7191346883773804, 0.5288378000259399, 0.999996542930603, 0.99964439868927, 0.9196721315383911, 0.16774530708789825, 0.9954264760017395, 0.9933912754058838, 0.940629780292511, 1.0, 0.9232226610183716, 0.5056142210960388, 0.7365531325340271, 0.9999996423721313, 0.21980130672454834, 0.8744170665740967], "n_positions_probed": 1, "per_restart_best": [3.346905469894409]}
+{"step": 375, "discrete_loss": 5.530508518218994, "best_sample_loss": 3.346905469894409, "soft_loss": 3.617955446243286, "best_discrete": 3.346905469894409, "best_soft": 2.6194920539855957, "best_argmax": 4.591865062713623, "best_sampling": 3.346905469894409, "relax_gap": 0.3458186648976744, "n_match": 8, "g_first_norm": 210.06788635253906, "vocab_size": 50257, "entropy": 0.7099907994270325, "entropy_per_token": [1.7428183555603027, 0.829892635345459, 1.0194450616836548, 1.0595176219940186, 0.8065576553344727, 0.7749965190887451, 5.122072616359219e-05, 0.003417948028072715, 0.32262173295021057, 2.6102969646453857, 0.03564498573541641, 0.04388827830553055, 0.2809465527534485, 1.895086398917556e-07, 0.41382235288619995, 0.001420126762241125, 0.8294057846069336, 6.714334631396923e-06, 2.953923225402832, 0.47114109992980957], "max_p": 0.7417277693748474, "max_p_per_token": [0.3452128767967224, 0.5372390151023865, 0.5235562324523926, 0.5639896392822266, 0.7134333252906799, 0.5409185290336609, 0.9999961853027344, 0.999620795249939, 0.9175630211830139, 0.14279495179653168, 0.9955644607543945, 0.9931742548942566, 0.9404494762420654, 1.0, 0.855956494808197, 0.9998793601989746, 0.744247555732727, 0.9999995231628418, 0.14039729535579681, 0.8805621862411499], "n_positions_probed": 1, "per_restart_best": [3.346905469894409]}
+{"step": 376, "discrete_loss": 5.736664772033691, "best_sample_loss": 4.112000942230225, "soft_loss": 3.6332740783691406, "best_discrete": 3.346905469894409, "best_soft": 2.6194920539855957, "best_argmax": 4.591865062713623, "best_sampling": 3.346905469894409, "relax_gap": 0.3666574180730597, "n_match": 8, "g_first_norm": 158.21310424804688, "vocab_size": 50257, "entropy": 0.7210086584091187, "entropy_per_token": [1.7273166179656982, 0.8231462836265564, 1.018899917602539, 1.0436043739318848, 0.7898374199867249, 0.7705361843109131, 5.381258961278945e-05, 0.0034189876168966293, 0.32842937111854553, 2.6824560165405273, 0.03355221822857857, 0.04490828514099121, 0.28115952014923096, 1.863546259528448e-07, 0.5606780052185059, 0.001647521392442286, 0.8654969930648804, 6.8565345827664714e-06, 2.983706474304199, 0.46131742000579834], "max_p": 0.7360646724700928, "max_p_per_token": [0.3672701120376587, 0.5034149289131165, 0.5309508442878723, 0.5807655453681946, 0.7203247547149658, 0.5387252569198608, 0.9999960660934448, 0.999620795249939, 0.9149783849716187, 0.12940487265586853, 0.9958577752113342, 0.9929814338684082, 0.9404186606407166, 1.0, 0.7530906796455383, 0.9998579025268555, 0.72447669506073, 0.9999995231628418, 0.14500156044960022, 0.8841572999954224], "n_positions_probed": 1, "per_restart_best": [3.346905469894409]}
+{"step": 377, "discrete_loss": 5.530508518218994, "best_sample_loss": 4.470524787902832, "soft_loss": 3.420677661895752, "best_discrete": 3.346905469894409, "best_soft": 2.6194920539855957, "best_argmax": 4.591865062713623, "best_sampling": 3.346905469894409, "relax_gap": 0.3814894867936443, "n_match": 8, "g_first_norm": 164.405517578125, "vocab_size": 50257, "entropy": 0.7283329367637634, "entropy_per_token": [1.712926983833313, 0.8120455741882324, 1.0187232494354248, 1.0311458110809326, 0.7836681604385376, 0.7655731439590454, 5.6577366194687784e-05, 0.0034073670394718647, 0.33292168378829956, 2.729721784591675, 0.03198588639497757, 0.04520134627819061, 0.28545060753822327, 1.8273463808782253e-07, 0.6845431327819824, 0.0019155730260536075, 0.9003835916519165, 6.886131359351566e-06, 2.970470905303955, 0.4565085470676422], "max_p": 0.7266305685043335, "max_p_per_token": [0.38557204604148865, 0.5022377967834473, 0.5355221629142761, 0.5930027365684509, 0.7197956442832947, 0.5406765341758728, 0.9999958276748657, 0.9996222257614136, 0.9127817749977112, 0.11453288048505783, 0.9960750937461853, 0.9929290413856506, 0.9392601251602173, 1.0, 0.5719689130783081, 0.9998321533203125, 0.7029639482498169, 0.9999995231628418, 0.13984763622283936, 0.8859948515892029], "n_positions_probed": 1, "per_restart_best": [3.346905469894409]}
+{"step": 378, "discrete_loss": 4.716332912445068, "best_sample_loss": 3.3234786987304688, "soft_loss": 2.9910061359405518, "best_discrete": 3.3234786987304688, "best_soft": 2.6194920539855957, "best_argmax": 4.591865062713623, "best_sampling": 3.3234786987304688, "relax_gap": 0.3658195484784094, "n_match": 9, "g_first_norm": 173.48345947265625, "vocab_size": 50257, "entropy": 0.7265110015869141, "entropy_per_token": [1.690497875213623, 0.8002885580062866, 1.020161509513855, 1.0196844339370728, 0.776190996170044, 0.7612060308456421, 5.838020661030896e-05, 0.0033707022666931152, 0.33306431770324707, 2.770479679107666, 0.030306275933980942, 0.04517802223563194, 0.30119889974594116, 1.7510301120182703e-07, 0.5867204666137695, 0.0022590607404708862, 0.9281914234161377, 6.7477776610758156e-06, 2.9998414516448975, 0.46151503920555115], "max_p": 0.7339116930961609, "max_p_per_token": [0.4045496881008148, 0.5119754672050476, 0.5353878736495972, 0.6035452485084534, 0.7183513641357422, 0.5404828786849976, 0.9999957084655762, 0.999626874923706, 0.9121325016021729, 0.10545007884502411, 0.9963052272796631, 0.9929350018501282, 0.9349139332771301, 1.0, 0.7277485132217407, 0.9997982382774353, 0.6824375987052917, 0.9999995231628418, 0.12814755737781525, 0.8844503164291382], "n_positions_probed": 1, "per_restart_best": [3.3234786987304688]}
+{"step": 379, "discrete_loss": 4.716332912445068, "best_sample_loss": 4.297903537750244, "soft_loss": 2.61267352104187, "best_discrete": 3.3234786987304688, "best_soft": 2.61267352104187, "best_argmax": 4.591865062713623, "best_sampling": 3.3234786987304688, "relax_gap": 0.44603708653649876, "n_match": 9, "g_first_norm": 153.92469787597656, "vocab_size": 50257, "entropy": 0.7414106130599976, "entropy_per_token": [1.6816083192825317, 0.788832426071167, 1.0204962491989136, 1.0235615968704224, 0.7918837070465088, 0.7584832906723022, 6.147653766674921e-05, 0.0034015595447272062, 0.3288422226905823, 2.794914722442627, 0.029230739921331406, 0.04558882862329483, 0.30426234006881714, 1.7101589833146136e-07, 0.5645464062690735, 0.0026350750122219324, 0.9385634660720825, 6.3818838498264086e-06, 2.9426076412200928, 0.8086856603622437], "max_p": 0.7227123975753784, "max_p_per_token": [0.4134247303009033, 0.5289345979690552, 0.5344637632369995, 0.6013495326042175, 0.7002155184745789, 0.5380342602729797, 0.9999954700469971, 0.99962317943573, 0.9132471680641174, 0.10181653499603271, 0.9964525699615479, 0.9928626418113708, 0.9340441823005676, 1.0, 0.7490355372428894, 0.9997603297233582, 0.6730666756629944, 0.9999996423721313, 0.14852184057235718, 0.6293991208076477], "n_positions_probed": 1, "per_restart_best": [3.3234786987304688]}
+{"step": 380, "discrete_loss": 4.538334369659424, "best_sample_loss": 3.372992753982544, "soft_loss": 2.64939546585083, "best_discrete": 3.3234786987304688, "best_soft": 2.61267352104187, "best_argmax": 4.538334369659424, "best_sampling": 3.3234786987304688, "relax_gap": 0.4162185396556287, "n_match": 9, "g_first_norm": 173.4591064453125, "vocab_size": 50257, "entropy": 0.7455412149429321, "entropy_per_token": [1.6574219465255737, 0.7848036885261536, 1.027637243270874, 1.0466829538345337, 0.8021978139877319, 0.7561063170433044, 6.413477240130305e-05, 0.0035471576265990734, 0.32357123494148254, 2.8065600395202637, 0.02870858460664749, 0.047124650329351425, 0.3191211223602295, 1.661146740161712e-07, 0.5312414169311523, 0.0030866966117173433, 0.9226705431938171, 6.312151072052075e-06, 3.0903961658477783, 0.7598767280578613], "max_p": 0.7242895364761353, "max_p_per_token": [0.44284588098526, 0.5399264693260193, 0.52772057056427, 0.5828855037689209, 0.6822826266288757, 0.5393012762069702, 0.999995231628418, 0.9996052384376526, 0.9147087335586548, 0.10356172174215317, 0.9965215921401978, 0.9925724267959595, 0.9298698306083679, 1.0, 0.777397871017456, 0.9997140765190125, 0.6801897883415222, 0.9999996423721313, 0.0941634476184845, 0.6825289130210876], "n_positions_probed": 1, "per_restart_best": [3.3234786987304688]}
+{"step": 381, "discrete_loss": 4.538334369659424, "best_sample_loss": 3.3234786987304688, "soft_loss": 2.7644519805908203, "best_discrete": 3.3234786987304688, "best_soft": 2.61267352104187, "best_argmax": 4.538334369659424, "best_sampling": 3.3234786987304688, "relax_gap": 0.390866393831119, "n_match": 9, "g_first_norm": 235.06544494628906, "vocab_size": 50257, "entropy": 0.724168062210083, "entropy_per_token": [1.6149781942367554, 0.7950950860977173, 1.041034460067749, 1.0276086330413818, 0.7887407541275024, 0.7510115504264832, 6.738967204000801e-05, 0.003593732602894306, 0.3194763660430908, 2.815770149230957, 0.0278928279876709, 0.047228094190359116, 0.32950031757354736, 1.6492801080403297e-07, 0.5013221502304077, 0.003591416869312525, 0.9312289953231812, 5.919263458054047e-06, 2.758592367172241, 0.7266231179237366], "max_p": 0.7321500182151794, "max_p_per_token": [0.46604397892951965, 0.49708184599876404, 0.5098121762275696, 0.5989899635314941, 0.6813129186630249, 0.5448107123374939, 0.9999949932098389, 0.9995993971824646, 0.9159159064292908, 0.10309859365224838, 0.9966327548027039, 0.9925754070281982, 0.9268888235092163, 1.0, 0.8000449538230896, 0.9996612071990967, 0.6746384501457214, 0.9999996423721313, 0.2241128832101822, 0.7117840051651001], "n_positions_probed": 1, "per_restart_best": [3.3234786987304688]}
+{"step": 382, "discrete_loss": 4.716332912445068, "best_sample_loss": 3.2244880199432373, "soft_loss": 2.8044087886810303, "best_discrete": 3.2244880199432373, "best_soft": 2.61267352104187, "best_argmax": 4.538334369659424, "best_sampling": 3.2244880199432373, "relax_gap": 0.4053836230939108, "n_match": 9, "g_first_norm": 174.56727600097656, "vocab_size": 50257, "entropy": 0.7390689253807068, "entropy_per_token": [1.70845627784729, 0.7853507995605469, 1.0438164472579956, 1.0492472648620605, 0.7855600118637085, 0.7492688298225403, 7.060276402626187e-05, 0.003598886076360941, 0.3194340467453003, 2.8191757202148438, 0.02785658836364746, 0.04684370383620262, 0.33953890204429626, 1.5921449403322185e-07, 0.46991610527038574, 0.004208979196846485, 0.9080194234848022, 5.864339982508682e-06, 3.0265419483184814, 0.694468080997467], "max_p": 0.7299134135246277, "max_p_per_token": [0.42555421590805054, 0.5623101592063904, 0.5048946142196655, 0.5818883180618286, 0.6721880435943604, 0.5448061227798462, 0.9999947547912598, 0.999599039554596, 0.9154614806175232, 0.11141606420278549, 0.9966314435005188, 0.992642343044281, 0.923997163772583, 1.0, 0.8215702176094055, 0.9995949864387512, 0.6857843399047852, 0.9999996423721313, 0.12265662103891373, 0.7372788786888123], "n_positions_probed": 1, "per_restart_best": [3.2244880199432373]}
+{"step": 383, "discrete_loss": 4.716332912445068, "best_sample_loss": 3.260146379470825, "soft_loss": 2.5860648155212402, "best_discrete": 3.2244880199432373, "best_soft": 2.5860648155212402, "best_argmax": 4.538334369659424, "best_sampling": 3.2244880199432373, "relax_gap": 0.4516789074203505, "n_match": 9, "g_first_norm": 169.8387908935547, "vocab_size": 50257, "entropy": 0.7344263195991516, "entropy_per_token": [1.6872599124908447, 0.7999118566513062, 1.0315757989883423, 1.094834566116333, 0.762277364730835, 0.7466029524803162, 7.490740972571075e-05, 0.00362400128506124, 0.31653571128845215, 2.8444201946258545, 0.026760924607515335, 0.04783089458942413, 0.34565386176109314, 1.574701116169308e-07, 0.45252543687820435, 0.004931807983666658, 0.9146023392677307, 5.590845375991194e-06, 2.9307124614715576, 0.678385317325592], "max_p": 0.7309927344322205, "max_p_per_token": [0.44222012162208557, 0.4978576600551605, 0.5196129679679871, 0.5824181437492371, 0.6811657547950745, 0.5419077277183533, 0.9999943971633911, 0.9995959401130676, 0.9162170886993408, 0.10457109659910202, 0.9967803955078125, 0.992466151714325, 0.9221751689910889, 1.0, 0.8326565027236938, 0.9995156526565552, 0.679731011390686, 0.9999996423721313, 0.16118429601192474, 0.7497849464416504], "n_positions_probed": 1, "per_restart_best": [3.2244880199432373]}
+{"step": 384, "discrete_loss": 4.722747325897217, "best_sample_loss": 3.248809814453125, "soft_loss": 2.6021409034729004, "best_discrete": 3.2244880199432373, "best_soft": 2.5860648155212402, "best_argmax": 4.538334369659424, "best_sampling": 3.2244880199432373, "relax_gap": 0.4490196650572341, "n_match": 9, "g_first_norm": 167.7391357421875, "vocab_size": 50257, "entropy": 0.7215027213096619, "entropy_per_token": [1.7667640447616577, 0.7929836511611938, 1.041184902191162, 1.1045796871185303, 0.25982943177223206, 0.7446531057357788, 7.89802725194022e-05, 0.0036639338359236717, 0.31565117835998535, 2.8520607948303223, 0.026455093175172806, 0.04826093465089798, 0.3502485752105713, 1.535493510118613e-07, 0.43493330478668213, 0.005801289342343807, 0.913676381111145, 5.522104402189143e-06, 3.1050972938537598, 0.6641253232955933], "max_p": 0.7412000298500061, "max_p_per_token": [0.40972134470939636, 0.5548588633537292, 0.5087854266166687, 0.5753017663955688, 0.9334657788276672, 0.5419236421585083, 0.9999940395355225, 0.9995912909507751, 0.9161226153373718, 0.10488392412662506, 0.9968197345733643, 0.9923852682113647, 0.9208341836929321, 1.0, 0.8433403968811035, 0.9994180202484131, 0.6763615012168884, 0.9999996423721313, 0.08967922627925873, 0.7605141997337341], "n_positions_probed": 1, "per_restart_best": [3.2244880199432373]}
+{"step": 385, "discrete_loss": 4.651482105255127, "best_sample_loss": 3.189018726348877, "soft_loss": 2.7977373600006104, "best_discrete": 3.189018726348877, "best_soft": 2.5860648155212402, "best_argmax": 4.538334369659424, "best_sampling": 3.189018726348877, "relax_gap": 0.3985277602509107, "n_match": 9, "g_first_norm": 238.9463348388672, "vocab_size": 50257, "entropy": 0.6959279179573059, "entropy_per_token": [1.6736775636672974, 0.7987542152404785, 1.0255098342895508, 1.0714874267578125, 0.2745486795902252, 0.6685128211975098, 8.290501136798412e-05, 0.003806428052484989, 0.320218026638031, 2.864011287689209, 0.025830823928117752, 0.048258379101753235, 0.3571045994758606, 1.5434174827078095e-07, 0.4416148066520691, 0.006701752543449402, 0.910598635673523, 5.227427664067363e-06, 2.7811412811279297, 0.6466928720474243], "max_p": 0.7567707300186157, "max_p_per_token": [0.4654439687728882, 0.5118352770805359, 0.5296604633331299, 0.5982875823974609, 0.9281898140907288, 0.6652392148971558, 0.9999938011169434, 0.9995731711387634, 0.914369523525238, 0.09577351808547974, 0.9969038367271423, 0.9924054741859436, 0.9187238812446594, 1.0, 0.839362382888794, 0.9993144273757935, 0.6817110180854797, 0.9999996423721313, 0.22599312663078308, 0.7726333737373352], "n_positions_probed": 1, "per_restart_best": [3.189018726348877]}
+{"step": 386, "discrete_loss": 6.9651408195495605, "best_sample_loss": 3.2463929653167725, "soft_loss": 2.7894961833953857, "best_discrete": 3.189018726348877, "best_soft": 2.5860648155212402, "best_argmax": 4.538334369659424, "best_sampling": 3.189018726348877, "relax_gap": 0.5995061326590976, "n_match": 9, "g_first_norm": 161.3004913330078, "vocab_size": 50257, "entropy": 0.7096403241157532, "entropy_per_token": [1.709582805633545, 0.7919489145278931, 1.022241473197937, 1.100903034210205, 0.28587430715560913, 0.6643170714378357, 8.6487882072106e-05, 0.00392025476321578, 0.3243166506290436, 2.8687357902526855, 0.026131168007850647, 0.04992125555872917, 0.3602209687232971, 1.5324893354318192e-07, 0.43152350187301636, 0.007871536538004875, 0.8849194645881653, 5.209851224208251e-06, 3.0407471656799316, 0.6195393204689026], "max_p": 0.7537480592727661, "max_p_per_token": [0.4563932418823242, 0.5463417172431946, 0.5378516912460327, 0.5736820101737976, 0.9239466786384583, 0.670953094959259, 0.9999934434890747, 0.9995589852333069, 0.9124714136123657, 0.10684783011674881, 0.9968578815460205, 0.9920951724052429, 0.917801558971405, 1.0, 0.845385730266571, 0.9991768002510071, 0.6955693364143372, 0.9999996423721313, 0.1106032133102417, 0.7894309759140015], "n_positions_probed": 1, "per_restart_best": [3.189018726348877]}
+{"step": 387, "discrete_loss": 4.705703258514404, "best_sample_loss": 3.1519784927368164, "soft_loss": 2.6051864624023438, "best_discrete": 3.1519784927368164, "best_soft": 2.5860648155212402, "best_argmax": 4.538334369659424, "best_sampling": 3.1519784927368164, "relax_gap": 0.4463768071884745, "n_match": 8, "g_first_norm": 174.55062866210938, "vocab_size": 50257, "entropy": 0.6988348960876465, "entropy_per_token": [1.638338327407837, 0.7940765619277954, 1.0265793800354004, 1.088158369064331, 0.2982521653175354, 0.6692240834236145, 9.153882274404168e-05, 0.0039760516956448555, 0.3315258324146271, 2.880411386489868, 0.02573569491505623, 0.05098551884293556, 0.3583459258079529, 1.5321597857109737e-07, 0.43485546112060547, 0.009211786091327667, 0.8963977694511414, 4.920381343254121e-06, 2.8668832778930664, 0.6036435961723328], "max_p": 0.7578476071357727, "max_p_per_token": [0.499732106924057, 0.520719587802887, 0.5314855575561523, 0.5804495215415955, 0.9192425608634949, 0.6645110249519348, 0.999993085861206, 0.9995519518852234, 0.909613847732544, 0.1042136400938034, 0.996908962726593, 0.9919096827507019, 0.9183167815208435, 1.0, 0.8434346914291382, 0.9990149736404419, 0.6893655061721802, 0.9999996423721313, 0.1894030123949051, 0.7990854978561401], "n_positions_probed": 1, "per_restart_best": [3.1519784927368164]}
+{"step": 388, "discrete_loss": 6.9651408195495605, "best_sample_loss": 3.178891897201538, "soft_loss": 2.6687426567077637, "best_discrete": 3.1519784927368164, "best_soft": 2.5860648155212402, "best_argmax": 4.538334369659424, "best_sampling": 3.1519784927368164, "relax_gap": 0.6168429719012698, "n_match": 8, "g_first_norm": 168.2078399658203, "vocab_size": 50257, "entropy": 0.7121137976646423, "entropy_per_token": [1.6917089223861694, 0.7904469966888428, 1.0256072282791138, 1.1094558238983154, 0.3092915415763855, 0.6661534309387207, 9.589049295755103e-05, 0.004093241412192583, 0.33700665831565857, 2.880922794342041, 0.026089034974575043, 0.052492473274469376, 0.35869958996772766, 1.5202967063032702e-07, 0.42714378237724304, 0.010842608287930489, 0.8859134316444397, 4.882308530795854e-06, 3.082200288772583, 0.5841068029403687], "max_p": 0.7536452412605286, "max_p_per_token": [0.4815945327281952, 0.5442404747009277, 0.5363353490829468, 0.5597092509269714, 0.9149038791656494, 0.6689862608909607, 0.9999927282333374, 0.999537467956543, 0.9071474075317383, 0.11012017726898193, 0.996856689453125, 0.9916263818740845, 0.9182326197624207, 1.0, 0.8479734659194946, 0.9988130331039429, 0.6933255195617676, 0.9999996423721313, 0.09328174591064453, 0.8102269172668457], "n_positions_probed": 1, "per_restart_best": [3.1519784927368164]}
+{"step": 389, "discrete_loss": 4.894839286804199, "best_sample_loss": 4.804945468902588, "soft_loss": 2.667837381362915, "best_discrete": 3.1519784927368164, "best_soft": 2.5860648155212402, "best_argmax": 4.538334369659424, "best_sampling": 3.1519784927368164, "relax_gap": 0.45496936159783, "n_match": 8, "g_first_norm": 224.78273010253906, "vocab_size": 50257, "entropy": 0.6485128402709961, "entropy_per_token": [1.58799147605896, 0.7878750562667847, 1.0311439037322998, 1.0725066661834717, 0.3205708861351013, 0.669275164604187, 0.0001020054696709849, 0.004169768653810024, 0.3443589210510254, 2.063373327255249, 0.02579142525792122, 0.052833572030067444, 0.3569917380809784, 1.5315134760385263e-07, 0.4286664128303528, 0.012582163326442242, 0.8947874307632446, 4.460621312318835e-06, 2.7445523738861084, 0.5726799368858337], "max_p": 0.7750856280326843, "max_p_per_token": [0.5359672904014587, 0.5339334011077881, 0.5287376046180725, 0.5851973295211792, 0.9104208946228027, 0.6644970178604126, 0.9999922513961792, 0.9995275735855103, 0.9041429162025452, 0.35439783334732056, 0.9968959093093872, 0.9915834069252014, 0.9186788201332092, 1.0, 0.8471019268035889, 0.9985927939414978, 0.6912831664085388, 0.9999997615814209, 0.2238996922969818, 0.816864013671875], "n_positions_probed": 1, "per_restart_best": [3.1519784927368164]}
+{"step": 390, "discrete_loss": 7.169186592102051, "best_sample_loss": 4.493802070617676, "soft_loss": 3.2044730186462402, "best_discrete": 3.1519784927368164, "best_soft": 2.5860648155212402, "best_argmax": 4.538334369659424, "best_sampling": 3.1519784927368164, "relax_gap": 0.5530213954569889, "n_match": 8, "g_first_norm": 195.66156005859375, "vocab_size": 50257, "entropy": 0.6847776770591736, "entropy_per_token": [1.6811414957046509, 0.79320228099823, 1.0252153873443604, 1.1117072105407715, 0.3296370804309845, 0.660344123840332, 0.00010172023758059368, 0.004081732593476772, 0.3473426401615143, 2.3415586948394775, 0.1204993799328804, 0.055897027254104614, 0.373033344745636, 1.5672847553105385e-07, 0.4219859838485718, 0.014664672315120697, 0.8648850917816162, 4.47105276180082e-06, 2.996708393096924, 0.5535423159599304], "max_p": 0.7677850723266602, "max_p_per_token": [0.49596527218818665, 0.5226238965988159, 0.5388633012771606, 0.5486214756965637, 0.9066332578659058, 0.6759857535362244, 0.9999922513961792, 0.9995388984680176, 0.902775228023529, 0.35155320167541504, 0.9781230092048645, 0.9909855127334595, 0.9137998819351196, 1.0, 0.8509858250617981, 0.9983218312263489, 0.7072448134422302, 0.9999997615814209, 0.14690950512886047, 0.8267785906791687], "n_positions_probed": 1, "per_restart_best": [3.1519784927368164]}
+{"step": 391, "discrete_loss": 4.894839286804199, "best_sample_loss": 3.3569602966308594, "soft_loss": 2.892815351486206, "best_discrete": 3.1519784927368164, "best_soft": 2.5860648155212402, "best_argmax": 4.538334369659424, "best_sampling": 3.1519784927368164, "relax_gap": 0.40900708236022565, "n_match": 8, "g_first_norm": 194.98841857910156, "vocab_size": 50257, "entropy": 0.6848546862602234, "entropy_per_token": [1.5579098463058472, 0.7814354300498962, 1.0240321159362793, 1.09522545337677, 0.34062373638153076, 0.6645115613937378, 0.00010343021858716384, 0.003920772112905979, 0.3649212121963501, 2.6346330642700195, 0.11971011757850647, 0.05640549957752228, 0.3749653398990631, 1.6066246644186322e-07, 0.4291326105594635, 0.017317142337560654, 0.8782358169555664, 4.218990852677962e-06, 2.812765598297119, 0.541240930557251], "max_p": 0.769024133682251, "max_p_per_token": [0.5556591153144836, 0.54220050573349, 0.5390883684158325, 0.5602108836174011, 0.9020549058914185, 0.6701868176460266, 0.9999921321868896, 0.9995589852333069, 0.8952370882034302, 0.23222464323043823, 0.9782686829566956, 0.9909249544143677, 0.9132094979286194, 1.0, 0.8468770980834961, 0.997965931892395, 0.7006648778915405, 0.9999997615814209, 0.2228953093290329, 0.833263099193573], "n_positions_probed": 1, "per_restart_best": [3.1519784927368164]}
+{"step": 392, "discrete_loss": 7.6329345703125, "best_sample_loss": 3.8304378986358643, "soft_loss": 2.76836895942688, "best_discrete": 3.1519784927368164, "best_soft": 2.5860648155212402, "best_argmax": 4.538334369659424, "best_sampling": 3.1519784927368164, "relax_gap": 0.6373126306893602, "n_match": 8, "g_first_norm": 172.08335876464844, "vocab_size": 50257, "entropy": 0.7227995991706848, "entropy_per_token": [1.6196706295013428, 0.7836368083953857, 1.0313793420791626, 1.1020152568817139, 0.3512420654296875, 0.646710216999054, 0.00010588707664282992, 0.003951262682676315, 0.3647152781486511, 2.7202200889587402, 0.1204998791217804, 0.058096110820770264, 0.7976080179214478, 1.6216138476465858e-07, 0.41530054807662964, 0.020263612270355225, 0.8576449155807495, 4.194514531263849e-06, 3.034456491470337, 0.5284719467163086], "max_p": 0.7426332831382751, "max_p_per_token": [0.5335532426834106, 0.5390159487724304, 0.5354779958724976, 0.5540251135826111, 0.897499680519104, 0.6926108002662659, 0.9999918937683105, 0.9995552897453308, 0.8950521945953369, 0.19460931420326233, 0.9780480861663818, 0.9905953407287598, 0.5202786922454834, 1.0, 0.8548229336738586, 0.9975588321685791, 0.711778461933136, 0.9999997615814209, 0.11836361140012741, 0.8398285508155823], "n_positions_probed": 1, "per_restart_best": [3.1519784927368164]}
+{"step": 393, "discrete_loss": 5.281790256500244, "best_sample_loss": 3.1519784927368164, "soft_loss": 2.744029998779297, "best_discrete": 3.1519784927368164, "best_soft": 2.5860648155212402, "best_argmax": 4.538334369659424, "best_sampling": 3.1519784927368164, "relax_gap": 0.4804735013090973, "n_match": 8, "g_first_norm": 187.96420288085938, "vocab_size": 50257, "entropy": 0.7128567695617676, "entropy_per_token": [1.578450083732605, 0.7866489887237549, 1.0415157079696655, 1.0913035869598389, 0.36164602637290955, 0.6644407510757446, 0.00011177662236150354, 0.004060214385390282, 0.3760986626148224, 2.7866106033325195, 0.12078079581260681, 0.058001939207315445, 0.8013258576393127, 1.80646858582989e-10, 0.41581273078918457, 0.023413456976413727, 0.8700937032699585, 3.946887773054186e-06, 2.7510998249053955, 0.5257177948951721], "max_p": 0.7445572018623352, "max_p_per_token": [0.5538156628608704, 0.5281926393508911, 0.5245572328567505, 0.5556389689445496, 0.8929663300514221, 0.6717426776885986, 0.9999912977218628, 0.9995414018630981, 0.8898369669914246, 0.16273048520088196, 0.9779489040374756, 0.9906438589096069, 0.5111809968948364, 1.0, 0.8545529842376709, 0.9971075654029846, 0.7062276005744934, 0.9999997615814209, 0.23323607444763184, 0.8412322402000427], "n_positions_probed": 1, "per_restart_best": [3.1519784927368164]}
+{"step": 394, "discrete_loss": 7.6329345703125, "best_sample_loss": 4.171685695648193, "soft_loss": 2.7819085121154785, "best_discrete": 3.1519784927368164, "best_soft": 2.5860648155212402, "best_argmax": 4.538334369659424, "best_sampling": 3.1519784927368164, "relax_gap": 0.6355387974979609, "n_match": 8, "g_first_norm": 154.9384307861328, "vocab_size": 50257, "entropy": 0.7257821559906006, "entropy_per_token": [1.6078393459320068, 0.7778489589691162, 1.0455896854400635, 1.0889885425567627, 0.3704373836517334, 0.6600793600082397, 0.0001163898705272004, 0.004208598751574755, 0.37392085790634155, 2.788135528564453, 0.12253692746162415, 0.06069641932845116, 0.8025288581848145, 1.8250470579239675e-10, 0.43088284134864807, 0.02729191444814205, 0.849761962890625, 3.885189926222665e-06, 2.976253032684326, 0.5285216569900513], "max_p": 0.7401841282844543, "max_p_per_token": [0.5429439544677734, 0.5589216947555542, 0.5254490971565247, 0.5568192005157471, 0.8889774084091187, 0.6782622933387756, 0.9999909400939941, 0.9995226860046387, 0.8903092741966248, 0.156159907579422, 0.9775011539459229, 0.9901305437088013, 0.49575939774513245, 1.0, 0.8591291904449463, 0.9965355396270752, 0.7174534797668457, 0.9999997615814209, 0.12957163155078888, 0.8402456045150757], "n_positions_probed": 1, "per_restart_best": [3.1519784927368164]}
+{"step": 395, "discrete_loss": 5.224825382232666, "best_sample_loss": 4.291576862335205, "soft_loss": 2.6341443061828613, "best_discrete": 3.1519784927368164, "best_soft": 2.5860648155212402, "best_argmax": 4.538334369659424, "best_sampling": 3.1519784927368164, "relax_gap": 0.49584070021929766, "n_match": 8, "g_first_norm": 173.782470703125, "vocab_size": 50257, "entropy": 0.7292844653129578, "entropy_per_token": [1.617884874343872, 0.7847648859024048, 1.0553596019744873, 1.0923064947128296, 0.3807406425476074, 0.6757970452308655, 0.00012236501788720489, 0.004347816109657288, 0.38362371921539307, 2.821467876434326, 0.1235588937997818, 0.06194823607802391, 0.8049442768096924, 1.8698724513210863e-10, 0.43463394045829773, 0.01630949229001999, 0.8703917264938354, 3.7127115319890436e-06, 2.929971933364868, 0.5275120139122009], "max_p": 0.7376964688301086, "max_p_per_token": [0.5400121212005615, 0.5485824346542358, 0.5141046643257141, 0.5467286109924316, 0.8842666745185852, 0.6587311625480652, 0.9999904632568359, 0.9995046854019165, 0.8856133818626404, 0.14497406780719757, 0.9772310256958008, 0.9899114966392517, 0.48850545287132263, 1.0, 0.8575962781906128, 0.9982362985610962, 0.7042865753173828, 0.9999997615814209, 0.17472225427627563, 0.8409311771392822], "n_positions_probed": 1, "per_restart_best": [3.1519784927368164]}
+{"step": 396, "discrete_loss": 6.9651408195495605, "best_sample_loss": 4.098783016204834, "soft_loss": 2.606628656387329, "best_discrete": 3.1519784927368164, "best_soft": 2.5860648155212402, "best_argmax": 4.538334369659424, "best_sampling": 3.1519784927368164, "relax_gap": 0.6257608103096614, "n_match": 8, "g_first_norm": 148.1633758544922, "vocab_size": 50257, "entropy": 0.7397680282592773, "entropy_per_token": [1.5985658168792725, 0.7793983817100525, 1.0517714023590088, 1.074576735496521, 0.39127689599990845, 0.6627321243286133, 0.00012888773926533759, 0.004455924965441227, 0.38395971059799194, 2.8389101028442383, 0.12442885339260101, 0.06460213661193848, 0.8044303059577942, 1.914811365022473e-10, 0.42291557788848877, 0.018607372418045998, 1.0132945775985718, 3.6009384984936332e-06, 3.0298190116882324, 0.5314827561378479], "max_p": 0.735871434211731, "max_p_per_token": [0.5499878525733948, 0.5615946650505066, 0.5216398239135742, 0.5615344643592834, 0.8793737292289734, 0.6767670512199402, 0.9999898672103882, 0.9994909763336182, 0.8851223587989807, 0.1395847201347351, 0.976986825466156, 0.989406168460846, 0.4956812560558319, 1.0, 0.8640089631080627, 0.9979530572891235, 0.6748178601264954, 0.9999997615814209, 0.10388434678316116, 0.8396050333976746], "n_positions_probed": 1, "per_restart_best": [3.1519784927368164]}
+{"step": 397, "discrete_loss": 4.705703258514404, "best_sample_loss": 3.1519784927368164, "soft_loss": 2.6374282836914062, "best_discrete": 3.1519784927368164, "best_soft": 2.5860648155212402, "best_argmax": 4.538334369659424, "best_sampling": 3.1519784927368164, "relax_gap": 0.43952515940751324, "n_match": 8, "g_first_norm": 190.21749877929688, "vocab_size": 50257, "entropy": 0.7283406257629395, "entropy_per_token": [1.5573772192001343, 0.7920900583267212, 1.0544673204421997, 1.0715079307556152, 0.40194910764694214, 0.6747892498970032, 0.00013722883886657655, 0.004604120273143053, 0.3923156261444092, 2.8756628036499023, 0.12456901371479034, 0.0653003677725792, 0.8054322004318237, 1.9822700425553563e-10, 0.42576539516448975, 0.021104417741298676, 1.0265216827392578, 1.0272701578273313e-09, 2.742846965789795, 0.5303716063499451], "max_p": 0.739210844039917, "max_p_per_token": [0.5685237050056458, 0.5238003134727478, 0.5174130201339722, 0.558344304561615, 0.8743115067481995, 0.6618635058403015, 0.9999892711639404, 0.9994717240333557, 0.8811340928077698, 0.13558192551136017, 0.976915717124939, 0.9893046617507935, 0.5016969442367554, 1.0, 0.8629773855209351, 0.99764084815979, 0.6662468910217285, 1.0, 0.22859036922454834, 0.8404108285903931], "n_positions_probed": 1, "per_restart_best": [3.1519784927368164]}
+{"step": 398, "discrete_loss": 5.6998467445373535, "best_sample_loss": 3.3639614582061768, "soft_loss": 2.735786199569702, "best_discrete": 3.1519784927368164, "best_soft": 2.5860648155212402, "best_argmax": 4.538334369659424, "best_sampling": 3.1519784927368164, "relax_gap": 0.5200246037857705, "n_match": 8, "g_first_norm": 154.3440704345703, "vocab_size": 50257, "entropy": 0.6721010208129883, "entropy_per_token": [1.5877844095230103, 0.7813421487808228, 1.0550181865692139, 1.074401617050171, 0.41077694296836853, 0.6682945489883423, 0.0001434293226338923, 0.00479006115347147, 0.38954517245292664, 2.8670272827148438, 0.1265581250190735, 0.06812871992588043, 0.8036367893218994, 1.9912133053523462e-10, 0.41257190704345703, 0.024208897724747658, 1.0112990140914917, 1.0079503898197117e-09, 1.6246010065078735, 0.5318928956985474], "max_p": 0.7623278498649597, "max_p_per_token": [0.5570608377456665, 0.5609548091888428, 0.52116858959198, 0.5566948056221008, 0.8699194192886353, 0.671528697013855, 0.9999886751174927, 0.9994478821754456, 0.8818880319595337, 0.14409326016902924, 0.9764009118080139, 0.988756000995636, 0.5147949457168579, 1.0, 0.8699390888214111, 0.9972423315048218, 0.6738935112953186, 1.0, 0.6226634383201599, 0.8401215076446533], "n_positions_probed": 1, "per_restart_best": [3.1519784927368164]}
+{"step": 399, "discrete_loss": 5.6998467445373535, "best_sample_loss": 3.644033193588257, "soft_loss": 4.0127668380737305, "best_discrete": 3.1519784927368164, "best_soft": 2.5860648155212402, "best_argmax": 4.538334369659424, "best_sampling": 3.1519784927368164, "relax_gap": 0.29598688913530785, "n_match": 8, "g_first_norm": 173.83935546875, "vocab_size": 50257, "entropy": 0.7103416323661804, "entropy_per_token": [1.5708706378936768, 0.7569454908370972, 1.068959355354309, 1.0806543827056885, 0.43176835775375366, 0.6678024530410767, 0.00014502316480502486, 0.00499432347714901, 0.38284456729888916, 2.873063087463379, 0.1266126036643982, 0.07123308628797531, 0.7975064516067505, 2.068534232790853e-10, 0.4035585820674896, 0.02772698551416397, 1.0122535228729248, 9.971182768353515e-10, 2.3077383041381836, 0.6221564412117004], "max_p": 0.7510477304458618, "max_p_per_token": [0.5693849325180054, 0.6197794675827026, 0.5074946284294128, 0.5656789541244507, 0.8592148423194885, 0.6735433340072632, 0.9999885559082031, 0.9994214773178101, 0.8845968246459961, 0.1507100909948349, 0.9763294458389282, 0.9881618618965149, 0.5418695211410522, 1.0, 0.874807596206665, 0.9967798590660095, 0.6779050827026367, 1.0, 0.332340270280838, 0.8029474020004272], "n_positions_probed": 1, "per_restart_best": [3.1519784927368164]}
+{"step": 400, "discrete_loss": 5.909275531768799, "best_sample_loss": 3.180107831954956, "soft_loss": 2.9995081424713135, "best_discrete": 3.1519784927368164, "best_soft": 2.5860648155212402, "best_argmax": 4.538334369659424, "best_sampling": 3.1519784927368164, "relax_gap": 0.4924067888955783, "n_match": 8, "g_first_norm": 159.21173095703125, "vocab_size": 50257, "entropy": 0.7386758327484131, "entropy_per_token": [1.6759871244430542, 0.7818589210510254, 1.095590353012085, 1.1012303829193115, 0.4405807852745056, 0.6831632852554321, 0.00014908568118698895, 0.0051040807738900185, 0.3911486864089966, 2.8733251094818115, 0.1276613175868988, 0.07100215554237366, 0.7959702014923096, 1.9821871921621437e-10, 0.4045126438140869, 0.03143775090575218, 1.018980622291565, 9.559407709858192e-10, 2.6465516090393066, 0.6292632818222046], "max_p": 0.7280256152153015, "max_p_per_token": [0.37595441937446594, 0.5874139070510864, 0.4739688038825989, 0.5441083908081055, 0.8544242978096008, 0.6541779041290283, 0.9999881982803345, 0.9994072914123535, 0.88050776720047, 0.1729753166437149, 0.9760069251060486, 0.9882498979568481, 0.538223385810852, 1.0, 0.8745030164718628, 0.9962865114212036, 0.6728973984718323, 1.0, 0.17057205736637115, 0.8008478879928589], "n_positions_probed": 1, "per_restart_best": [3.1519784927368164]}
+{"step": 401, "discrete_loss": 5.6998467445373535, "best_sample_loss": 3.2932803630828857, "soft_loss": 2.8492202758789062, "best_discrete": 3.1519784927368164, "best_soft": 2.5860648155212402, "best_argmax": 4.538334369659424, "best_sampling": 3.1519784927368164, "relax_gap": 0.5001233535604171, "n_match": 8, "g_first_norm": 170.8018035888672, "vocab_size": 50257, "entropy": 0.7402505278587341, "entropy_per_token": [1.67171311378479, 0.8130147457122803, 1.0763330459594727, 1.090261459350586, 0.4491550922393799, 0.6480042934417725, 0.00015544769121333957, 0.0051949480548501015, 0.39207738637924194, 2.8898282051086426, 0.1275814324617386, 0.07665673643350601, 0.7928889989852905, 1.9924974170582033e-10, 0.4070214033126831, 0.03598228469491005, 0.9939819574356079, 9.159061842289873e-10, 2.6937458515167236, 0.6414139270782471], "max_p": 0.7300600409507751, "max_p_per_token": [0.38975176215171814, 0.5222616195678711, 0.49750351905822754, 0.5505305528640747, 0.8497462272644043, 0.6978280544281006, 0.9999877214431763, 0.9993951320648193, 0.8798993825912476, 0.1422063410282135, 0.9759395122528076, 0.9871373176574707, 0.5488407611846924, 1.0, 0.8737832307815552, 0.9956629872322083, 0.6897841691970825, 1.0, 0.20465101301670074, 0.796291708946228], "n_positions_probed": 1, "per_restart_best": [3.1519784927368164]}
+{"step": 402, "discrete_loss": 5.6998467445373535, "best_sample_loss": 3.128864049911499, "soft_loss": 2.6901307106018066, "best_discrete": 3.128864049911499, "best_soft": 2.5860648155212402, "best_argmax": 4.538334369659424, "best_sampling": 3.128864049911499, "relax_gap": 0.5280345540553372, "n_match": 8, "g_first_norm": 129.8428497314453, "vocab_size": 50257, "entropy": 0.7546108365058899, "entropy_per_token": [1.7102185487747192, 0.8041893243789673, 1.1103034019470215, 1.0941176414489746, 0.459560751914978, 0.6519315242767334, 0.0001631696941331029, 0.005287309177219868, 0.3926553428173065, 2.8939826488494873, 0.12821711599826813, 0.0808442160487175, 0.7898807525634766, 1.969289453729317e-10, 0.4070122539997101, 0.04166262596845627, 0.9937378764152527, 8.603815437879803e-10, 2.8827414512634277, 0.6457109451293945], "max_p": 0.726848304271698, "max_p_per_token": [0.38826194405555725, 0.5544880628585815, 0.4656730890274048, 0.5372105836868286, 0.8439085483551025, 0.6943300366401672, 0.999987006187439, 0.9993832111358643, 0.8792955875396729, 0.14656154811382294, 0.9757245182991028, 0.9863102436065674, 0.5563958883285522, 1.0, 0.8741940855979919, 0.994866132736206, 0.6899282336235046, 1.0, 0.1550450623035431, 0.7954031825065613], "n_positions_probed": 1, "per_restart_best": [3.128864049911499]}
+{"step": 403, "discrete_loss": 5.5937113761901855, "best_sample_loss": 3.1891210079193115, "soft_loss": 2.602785110473633, "best_discrete": 3.128864049911499, "best_soft": 2.5860648155212402, "best_argmax": 4.538334369659424, "best_sampling": 3.128864049911499, "relax_gap": 0.5346944210327919, "n_match": 8, "g_first_norm": 153.45138549804688, "vocab_size": 50257, "entropy": 0.7313123345375061, "entropy_per_token": [1.6776942014694214, 0.8074299097061157, 1.0664511919021606, 0.6408511400222778, 0.4691005349159241, 0.6498762369155884, 0.00017350117559544742, 0.005336514208465815, 0.394260436296463, 2.903393268585205, 0.12846827507019043, 0.08691859245300293, 0.787240743637085, 1.9939910833599583e-10, 0.4086143374443054, 0.04833144694566727, 0.9929565787315369, 7.97291010989909e-10, 2.904616117477417, 0.6545332670211792], "max_p": 0.7443527579307556, "max_p_per_token": [0.42483845353126526, 0.5389410853385925, 0.5192883014678955, 0.8171641230583191, 0.8384778499603271, 0.6974166035652161, 0.9999860525131226, 0.9993767142295837, 0.8783147931098938, 0.13041232526302338, 0.9755911827087402, 0.98508620262146, 0.5653918981552124, 1.0, 0.8739281296730042, 0.9939022064208984, 0.6919910311698914, 1.0, 0.16451002657413483, 0.7924379706382751], "n_positions_probed": 1, "per_restart_best": [3.128864049911499]}
+{"step": 404, "discrete_loss": 4.794850826263428, "best_sample_loss": 3.128863573074341, "soft_loss": 2.7095253467559814, "best_discrete": 3.128863573074341, "best_soft": 2.5860648155212402, "best_argmax": 4.538334369659424, "best_sampling": 3.128863573074341, "relax_gap": 0.4349093548615185, "n_match": 8, "g_first_norm": 139.82803344726562, "vocab_size": 50257, "entropy": 0.7546564936637878, "entropy_per_token": [1.6845206022262573, 0.8012908697128296, 1.102396011352539, 0.6911057233810425, 0.7605591416358948, 0.6735868453979492, 0.000187106488738209, 0.005413809325546026, 0.3987826704978943, 2.921527624130249, 0.13083234429359436, 0.09056422859430313, 0.783748984336853, 2.0440253656328622e-10, 0.4032217264175415, 0.054916832596063614, 0.9955991506576538, 7.449242334089945e-10, 2.9337780475616455, 0.6610981822013855], "max_p": 0.7275762557983398, "max_p_per_token": [0.433383047580719, 0.5516785979270935, 0.4730828106403351, 0.7957041263580322, 0.5871061682701111, 0.6714798212051392, 0.999984860420227, 0.9993667006492615, 0.8758628368377686, 0.12253541499376297, 0.974951982498169, 0.9843283891677856, 0.5723902583122253, 1.0, 0.8768814206123352, 0.9929236173629761, 0.6938551664352417, 1.0, 0.1549476683139801, 0.7910614013671875], "n_positions_probed": 1, "per_restart_best": [3.128863573074341]}
+{"step": 405, "discrete_loss": 4.908740520477295, "best_sample_loss": 3.1589126586914062, "soft_loss": 2.6597723960876465, "best_discrete": 3.128863573074341, "best_soft": 2.5860648155212402, "best_argmax": 4.538334369659424, "best_sampling": 3.128863573074341, "relax_gap": 0.4581558375326331, "n_match": 8, "g_first_norm": 155.48236083984375, "vocab_size": 50257, "entropy": 0.7581081390380859, "entropy_per_token": [1.6768887042999268, 0.8132820129394531, 1.0998520851135254, 0.7320609092712402, 0.7648087739944458, 0.6650117635726929, 0.00020220581791363657, 0.005509324371814728, 0.3978733420372009, 2.9340901374816895, 0.13154052197933197, 0.09236133843660355, 0.7792097330093384, 2.1086757340249562e-10, 0.3916456997394562, 0.06272520124912262, 1.0084803104400635, 7.145209424130883e-10, 2.9318623542785645, 0.6747581362724304], "max_p": 0.7244880795478821, "max_p_per_token": [0.4427737295627594, 0.5024521946907043, 0.4765061140060425, 0.7775072455406189, 0.5664659142494202, 0.6834913492202759, 0.999983549118042, 0.9993543028831482, 0.8759944438934326, 0.11685141921043396, 0.974709153175354, 0.9839679002761841, 0.5834973454475403, 1.0, 0.8825327754020691, 0.9917312264442444, 0.6850757002830505, 1.0, 0.16055932641029358, 0.7863079309463501], "n_positions_probed": 1, "per_restart_best": [3.128863573074341]}
+{"step": 406, "discrete_loss": 6.002975940704346, "best_sample_loss": 3.1657581329345703, "soft_loss": 2.6216511726379395, "best_discrete": 3.128863573074341, "best_soft": 2.5860648155212402, "best_argmax": 4.538334369659424, "best_sampling": 3.128863573074341, "relax_gap": 0.563274749302038, "n_match": 8, "g_first_norm": 138.05589294433594, "vocab_size": 50257, "entropy": 0.7664250731468201, "entropy_per_token": [1.6894519329071045, 0.8055440187454224, 1.118101954460144, 0.7714792490005493, 0.7612330317497253, 0.666003406047821, 0.010064680129289627, 0.005550120025873184, 0.3984290659427643, 2.9409217834472656, 0.1324920356273651, 0.09336555004119873, 0.7729496955871582, 2.159290662939739e-10, 0.3825470507144928, 0.07213578373193741, 1.0253289937973022, 6.844848021714256e-10, 2.9982848167419434, 0.6846187710762024], "max_p": 0.721551775932312, "max_p_per_token": [0.437373548746109, 0.5268465280532837, 0.4446731507778168, 0.7589903473854065, 0.5612644553184509, 0.683499813079834, 0.9986825585365295, 0.9993494153022766, 0.8754309415817261, 0.11382609605789185, 0.9744167923927307, 0.9837707281112671, 0.5966495275497437, 1.0, 0.8869115114212036, 0.9902542233467102, 0.6738868951797485, 1.0, 0.14179816842079163, 0.7834106683731079], "n_positions_probed": 1, "per_restart_best": [3.128863573074341]}
+{"step": 407, "discrete_loss": 4.908740520477295, "best_sample_loss": 3.223088502883911, "soft_loss": 2.6110188961029053, "best_discrete": 3.128863573074341, "best_soft": 2.5860648155212402, "best_argmax": 4.538334369659424, "best_sampling": 3.128863573074341, "relax_gap": 0.46808781494748347, "n_match": 8, "g_first_norm": 174.32144165039062, "vocab_size": 50257, "entropy": 0.7568881511688232, "entropy_per_token": [1.6403926610946655, 0.8099955320358276, 1.0595892667770386, 0.8174711465835571, 0.7592111229896545, 0.6627013683319092, 0.010966410860419273, 0.0057168942876160145, 0.3993343710899353, 2.951603889465332, 0.13390743732452393, 0.09478536993265152, 0.7653375864028931, 2.2440022062752973e-10, 0.3768155872821808, 0.08238379657268524, 1.038254976272583, 6.557614451452309e-10, 2.834071159362793, 0.6952245235443115], "max_p": 0.7264302372932434, "max_p_per_token": [0.4594977796077728, 0.48766106367111206, 0.5291135907173157, 0.7363690137863159, 0.5493613481521606, 0.6883492469787598, 0.9985456466674805, 0.9993314743041992, 0.8747613430023193, 0.11104273796081543, 0.9739994406700134, 0.9834967851638794, 0.6112942695617676, 1.0, 0.8897455334663391, 0.9885954260826111, 0.6655327081680298, 1.0, 0.20162303745746613, 0.7802832126617432], "n_positions_probed": 1, "per_restart_best": [3.128863573074341]}
+{"step": 408, "discrete_loss": 5.726365089416504, "best_sample_loss": 3.203202962875366, "soft_loss": 2.6650002002716064, "best_discrete": 3.128863573074341, "best_soft": 2.5860648155212402, "best_argmax": 4.538334369659424, "best_sampling": 3.128863573074341, "relax_gap": 0.5346087511609986, "n_match": 8, "g_first_norm": 142.463623046875, "vocab_size": 50257, "entropy": 0.7729396224021912, "entropy_per_token": [1.7102874517440796, 0.805914044380188, 1.0818707942962646, 0.8462966084480286, 0.7548835277557373, 0.6707431674003601, 0.011727931909263134, 0.005802695639431477, 0.40069928765296936, 2.93037748336792, 0.13488350808620453, 0.09418578445911407, 0.7532416582107544, 2.2666861443365605e-10, 0.36558887362480164, 0.09494657069444656, 1.0447375774383545, 6.364176408091282e-10, 3.049978256225586, 0.7026259899139404], "max_p": 0.7220184206962585, "max_p_per_token": [0.429519921541214, 0.5214499235153198, 0.5028680562973022, 0.7212783694267273, 0.5500362515449524, 0.6802985072135925, 0.998428463935852, 0.9993209838867188, 0.8737198710441589, 0.1275959461927414, 0.9737108945846558, 0.983627438545227, 0.6307700276374817, 1.0, 0.8948756456375122, 0.9865012764930725, 0.6608484983444214, 1.0, 0.12663210928440094, 0.7788864374160767], "n_positions_probed": 1, "per_restart_best": [3.128863573074341]}
+{"step": 409, "discrete_loss": 4.794850826263428, "best_sample_loss": 4.73059606552124, "soft_loss": 2.6238808631896973, "best_discrete": 3.128863573074341, "best_soft": 2.5860648155212402, "best_argmax": 4.538334369659424, "best_sampling": 3.128863573074341, "relax_gap": 0.45277111671178777, "n_match": 8, "g_first_norm": 203.48123168945312, "vocab_size": 50257, "entropy": 0.6937412619590759, "entropy_per_token": [1.6569453477859497, 0.8007202744483948, 1.096382975578308, 0.8980250358581543, 0.7510344982147217, 0.6668273210525513, 0.012784114107489586, 0.005854930263012648, 0.40276047587394714, 1.7095623016357422, 0.13610725104808807, 0.09476011991500854, 0.7384771108627319, 2.3558299755421785e-10, 0.36183083057403564, 0.10892613232135773, 1.0483192205429077, 6.04525152159141e-10, 2.678598642349243, 0.7069081664085388], "max_p": 0.7535936236381531, "max_p_per_token": [0.4526430368423462, 0.521893322467804, 0.4814903736114502, 0.6927672028541565, 0.546402633190155, 0.6855330467224121, 0.9982637763023376, 0.9993140697479248, 0.8724517822265625, 0.6343787908554077, 0.9733514785766602, 0.9835479855537415, 0.6519744992256165, 1.0, 0.896746814250946, 0.9840947985649109, 0.6613457798957825, 1.0, 0.2569533884525299, 0.778719425201416], "n_positions_probed": 1, "per_restart_best": [3.128863573074341]}
+{"step": 410, "discrete_loss": 7.230386257171631, "best_sample_loss": 4.477295398712158, "soft_loss": 3.492448091506958, "best_discrete": 3.128863573074341, "best_soft": 2.5860648155212402, "best_argmax": 4.538334369659424, "best_sampling": 3.128863573074341, "relax_gap": 0.5169762766072297, "n_match": 8, "g_first_norm": 183.93264770507812, "vocab_size": 50257, "entropy": 0.7586202621459961, "entropy_per_token": [1.7765871286392212, 0.791700541973114, 1.1235368251800537, 0.9374703764915466, 0.7513189911842346, 0.6729061007499695, 0.01302667148411274, 0.005534523632377386, 0.41732460260391235, 2.5422534942626953, 0.13990016281604767, 0.0963844358921051, 0.7579343318939209, 2.4119384267606847e-10, 0.35213398933410645, 0.12389706075191498, 1.0246883630752563, 5.949683523631677e-10, 2.9180827140808105, 0.7277251482009888], "max_p": 0.7282477617263794, "max_p_per_token": [0.3912774324417114, 0.5528327822685242, 0.444144606590271, 0.6681179404258728, 0.5434955358505249, 0.6790333390235901, 0.9982255101203918, 0.9993563294410706, 0.864898145198822, 0.33056148886680603, 0.9724544286727905, 0.9832339286804199, 0.6344903707504272, 1.0, 0.9011136293411255, 0.9814360737800598, 0.67607182264328, 1.0, 0.17325305938720703, 0.770957887172699], "n_positions_probed": 1, "per_restart_best": [3.128863573074341]}
+{"step": 411, "discrete_loss": 4.684231758117676, "best_sample_loss": 3.128864049911499, "soft_loss": 2.9191784858703613, "best_discrete": 3.128863573074341, "best_soft": 2.5860648155212402, "best_argmax": 4.538334369659424, "best_sampling": 3.128863573074341, "relax_gap": 0.37680741760663616, "n_match": 8, "g_first_norm": 231.53602600097656, "vocab_size": 50257, "entropy": 0.7618398070335388, "entropy_per_token": [1.6866469383239746, 0.8065377473831177, 1.001708745956421, 0.9685295820236206, 0.7482354640960693, 0.6661014556884766, 0.01365822833031416, 0.005570105277001858, 0.4315088093280792, 2.7716879844665527, 0.14193357527256012, 0.10170024633407593, 0.7595800161361694, 2.469046911368622e-10, 0.35778093338012695, 0.14333437383174896, 1.0311970710754395, 5.859007723429954e-10, 2.861142873764038, 0.7399425506591797], "max_p": 0.7290776371955872, "max_p_per_token": [0.44924649596214294, 0.49980005621910095, 0.5909682512283325, 0.6483669281005859, 0.5245932936668396, 0.6870234608650208, 0.9981254935264587, 0.999351441860199, 0.8573938012123108, 0.2057720571756363, 0.9718814492225647, 0.982272744178772, 0.6371692419052124, 1.0, 0.8990800976753235, 0.9778453707695007, 0.6701099276542664, 1.0, 0.2152348756790161, 0.7673180103302002], "n_positions_probed": 1, "per_restart_best": [3.128863573074341]}
+{"step": 412, "discrete_loss": 5.830402374267578, "best_sample_loss": 3.8991494178771973, "soft_loss": 2.6767985820770264, "best_discrete": 3.128863573074341, "best_soft": 2.5860648155212402, "best_argmax": 4.538334369659424, "best_sampling": 3.128863573074341, "relax_gap": 0.540889562975782, "n_match": 8, "g_first_norm": 148.33465576171875, "vocab_size": 50257, "entropy": 0.7772520184516907, "entropy_per_token": [1.7102612257003784, 0.803910493850708, 1.0202927589416504, 0.9869070053100586, 0.7396301031112671, 0.6610932946205139, 0.014543937519192696, 0.005654972977936268, 0.4320994019508362, 2.8199667930603027, 0.14201954007148743, 0.10419807583093643, 0.7458950281143188, 2.5075069798319305e-10, 0.34763315320014954, 0.16527405381202698, 1.0338454246520996, 5.654994805759372e-10, 3.0639376640319824, 0.7478767037391663], "max_p": 0.7223482728004456, "max_p_per_token": [0.43861815333366394, 0.5001868009567261, 0.5740144848823547, 0.63541579246521, 0.5395936369895935, 0.6931514143943787, 0.9979836940765381, 0.999340832233429, 0.8567244410514832, 0.16460280120372772, 0.9718030691146851, 0.981777012348175, 0.6563587188720703, 1.0, 0.9035084843635559, 0.973645806312561, 0.6707031726837158, 1.0, 0.1237475797533989, 0.7657890319824219], "n_positions_probed": 1, "per_restart_best": [3.128863573074341]}
+{"step": 413, "discrete_loss": 4.794850826263428, "best_sample_loss": 4.373077869415283, "soft_loss": 2.6180005073547363, "best_discrete": 3.128863573074341, "best_soft": 2.5860648155212402, "best_argmax": 4.538334369659424, "best_sampling": 3.128863573074341, "relax_gap": 0.4539975064469495, "n_match": 8, "g_first_norm": 209.94703674316406, "vocab_size": 50257, "entropy": 0.7605155110359192, "entropy_per_token": [1.6472365856170654, 0.7972779273986816, 1.0463091135025024, 1.0165988206863403, 0.7329857349395752, 0.6560436487197876, 0.015747521072626114, 0.005717678461223841, 0.4365162253379822, 2.8547780513763428, 0.14326617121696472, 0.10652758926153183, 0.7285647392272949, 1.919823411355992e-09, 0.3450731933116913, 0.1900397688150406, 1.0363671779632568, 5.268971925431742e-10, 2.701361656188965, 0.7498986124992371], "max_p": 0.728486955165863, "max_p_per_token": [0.4667600691318512, 0.5059307217597961, 0.5476459264755249, 0.6135238409042358, 0.5506277680397034, 0.6988155841827393, 0.9977884292602539, 0.9993324875831604, 0.854230523109436, 0.14223192632198334, 0.9714216589927673, 0.9813655614852905, 0.6776241064071655, 1.0, 0.904784083366394, 0.968722939491272, 0.6738331913948059, 1.0, 0.24842721223831177, 0.7666714191436768], "n_positions_probed": 1, "per_restart_best": [3.128863573074341]}
+{"step": 414, "discrete_loss": 5.726365089416504, "best_sample_loss": 4.492292404174805, "soft_loss": 2.6937530040740967, "best_discrete": 3.128863573074341, "best_soft": 2.5860648155212402, "best_argmax": 4.538334369659424, "best_sampling": 3.128863573074341, "relax_gap": 0.529587624608025, "n_match": 8, "g_first_norm": 138.56369018554688, "vocab_size": 50257, "entropy": 0.7769818305969238, "entropy_per_token": [1.7109557390213013, 0.7962515354156494, 1.0643582344055176, 1.0250147581100464, 0.728705883026123, 0.6628722548484802, 0.01671903394162655, 0.005836612079292536, 0.43085777759552, 2.8386287689208984, 0.1427653431892395, 0.10618089139461517, 0.7122182846069336, 1.9152901487018426e-09, 0.33377528190612793, 0.21855492889881134, 1.0362157821655273, 5.152432369648352e-10, 2.957951068878174, 0.7517741918563843], "max_p": 0.7231811285018921, "max_p_per_token": [0.43962278962135315, 0.5243384838104248, 0.5289076566696167, 0.6060973405838013, 0.5566281676292419, 0.6922400593757629, 0.9976288676261902, 0.9993174076080322, 0.856730580329895, 0.1492568552494049, 0.9714968204498291, 0.9814994931221008, 0.6951456665992737, 1.0, 0.9094793200492859, 0.9628222584724426, 0.6745900511741638, 1.0, 0.14980337023735046, 0.7680175304412842], "n_positions_probed": 1, "per_restart_best": [3.128863573074341]}
+{"step": 415, "discrete_loss": 4.869917392730713, "best_sample_loss": 4.119686126708984, "soft_loss": 2.5147221088409424, "best_discrete": 3.128863573074341, "best_soft": 2.5147221088409424, "best_argmax": 4.538334369659424, "best_sampling": 3.128863573074341, "relax_gap": 0.4836211980526306, "n_match": 8, "g_first_norm": 160.39674377441406, "vocab_size": 50257, "entropy": 0.7855899333953857, "entropy_per_token": [1.731888771057129, 0.797287106513977, 1.0821634531021118, 1.032930850982666, 0.726115345954895, 0.6636278629302979, 0.017836574465036392, 0.005995858460664749, 0.4281887412071228, 2.8542261123657227, 0.14271490275859833, 0.10885182023048401, 0.6916642189025879, 1.9035844012194048e-09, 0.3294374942779541, 0.25700706243515015, 1.0495226383209229, 4.98992958064548e-10, 3.0380606651306152, 0.7542796730995178], "max_p": 0.7207505106925964, "max_p_per_token": [0.43245717883110046, 0.5160934925079346, 0.5076318979263306, 0.5989221334457397, 0.5525059103965759, 0.6924358010292053, 0.9974429607391357, 0.9992966651916504, 0.8576875329017639, 0.1502898633480072, 0.9714535474777222, 0.9810066223144531, 0.7155359983444214, 1.0, 0.9113789200782776, 0.9549351930618286, 0.6671358942985535, 1.0, 0.14020797610282898, 0.768592894077301], "n_positions_probed": 1, "per_restart_best": [3.128863573074341]}
+{"step": 416, "discrete_loss": 4.869917392730713, "best_sample_loss": 3.128864049911499, "soft_loss": 2.477175235748291, "best_discrete": 3.128863573074341, "best_soft": 2.477175235748291, "best_argmax": 4.538334369659424, "best_sampling": 3.128863573074341, "relax_gap": 0.49133115903650626, "n_match": 8, "g_first_norm": 171.51878356933594, "vocab_size": 50257, "entropy": 0.7247966527938843, "entropy_per_token": [1.688886046409607, 0.7906434535980225, 1.0890686511993408, 1.0372629165649414, 0.7213419079780579, 0.6529340147972107, 0.019265204668045044, 0.0060501196421682835, 0.4255039691925049, 2.865147352218628, 0.14340122044086456, 0.11226412653923035, 0.6699899435043335, 1.9176125132247535e-09, 0.3243858516216278, 0.29498329758644104, 0.007029273547232151, 4.737180092639903e-10, 2.8933186531066895, 0.7544560432434082], "max_p": 0.7427384257316589, "max_p_per_token": [0.4513276219367981, 0.523200511932373, 0.4985285997390747, 0.5934344530105591, 0.5572887659072876, 0.7043007612228394, 0.9972019195556641, 0.9992896318435669, 0.8587933778762817, 0.16697552800178528, 0.9712178707122803, 0.9803884029388428, 0.734725296497345, 1.0, 0.9135136604309082, 0.9463548064231873, 0.9991716146469116, 1.0, 0.18892262876033783, 0.7701320648193359], "n_positions_probed": 1, "per_restart_best": [3.128863573074341]}
+{"step": 417, "discrete_loss": 5.726365089416504, "best_sample_loss": 4.791863918304443, "soft_loss": 2.5616772174835205, "best_discrete": 3.128863573074341, "best_soft": 2.477175235748291, "best_argmax": 4.538334369659424, "best_sampling": 3.128863573074341, "relax_gap": 0.552652131416136, "n_match": 8, "g_first_norm": 145.29852294921875, "vocab_size": 50257, "entropy": 0.7384592294692993, "entropy_per_token": [1.7582409381866455, 0.7894701361656189, 1.1000300645828247, 1.042145848274231, 0.7206056118011475, 0.6566261649131775, 0.020653842017054558, 0.0060363165102899075, 0.42001885175704956, 2.8594348430633545, 0.14201918244361877, 0.11302002519369125, 0.6488006114959717, 1.885708700299915e-09, 0.31826502084732056, 0.34157612919807434, 0.0072056944482028484, 4.5540304860480774e-10, 3.0638067722320557, 0.7612277269363403], "max_p": 0.7373819351196289, "max_p_per_token": [0.4180870056152344, 0.5270501375198364, 0.48384085297584534, 0.5882763862609863, 0.5495548248291016, 0.7014209628105164, 0.996964156627655, 0.9992923736572266, 0.8611335754394531, 0.1728990375995636, 0.9715253114700317, 0.9803430438041687, 0.7517476677894592, 1.0, 0.9160286784172058, 0.9353010058403015, 0.9991520643234253, 1.0, 0.12632471323013306, 0.7686969637870789], "n_positions_probed": 1, "per_restart_best": [3.128863573074341]}
+{"step": 418, "discrete_loss": 5.360703945159912, "best_sample_loss": 2.9096896648406982, "soft_loss": 2.518314838409424, "best_discrete": 2.9096896648406982, "best_soft": 2.477175235748291, "best_argmax": 4.538334369659424, "best_sampling": 2.9096896648406982, "relax_gap": 0.5302268388309026, "n_match": 9, "g_first_norm": 182.80760192871094, "vocab_size": 50257, "entropy": 0.6027389764785767, "entropy_per_token": [1.7103182077407837, 0.7883275747299194, 1.072009563446045, 1.0493412017822266, 0.7189324498176575, 0.646580159664154, 0.022515757009387016, 0.005965080112218857, 0.41967087984085083, 2.869023323059082, 0.1422814130783081, 0.11629274487495422, 0.6261254549026489, 1.88822690816437e-09, 0.3181009292602539, 0.39418068528175354, 0.007189389318227768, 4.314288370999009e-10, 0.3838360011577606, 0.7640885710716248], "max_p": 0.7808052897453308, "max_p_per_token": [0.4382684826850891, 0.5114433169364929, 0.5177545547485352, 0.5800220370292664, 0.5455180406570435, 0.7120817303657532, 0.9966403245925903, 0.9993020296096802, 0.8610722422599792, 0.18741174042224884, 0.971401572227478, 0.9798054099082947, 0.7688256502151489, 1.0, 0.9163062572479248, 0.9221464991569519, 0.9991580247879028, 1.0, 0.9401920437812805, 0.7687557935714722], "n_positions_probed": 1, "per_restart_best": [2.9096896648406982]}
+{"step": 419, "discrete_loss": 5.360703945159912, "best_sample_loss": 4.131067276000977, "soft_loss": 4.338393211364746, "best_discrete": 2.9096896648406982, "best_soft": 2.477175235748291, "best_argmax": 4.538334369659424, "best_sampling": 2.9096896648406982, "relax_gap": 0.19070456870094327, "n_match": 9, "g_first_norm": 173.57000732421875, "vocab_size": 50257, "entropy": 0.61134272813797, "entropy_per_token": [1.7327369451522827, 0.7901396155357361, 1.0798829793930054, 1.0571789741516113, 0.7201714515686035, 0.6553858518600464, 0.02330293133854866, 0.006192185450345278, 0.400144100189209, 2.8377761840820312, 0.14933134615421295, 0.11282730102539062, 0.5974459648132324, 2.0658639243720245e-09, 0.3135075569152832, 0.45860418677330017, 0.007103569805622101, 4.2727674176568087e-10, 0.4416006803512573, 0.8435226082801819], "max_p": 0.7793705463409424, "max_p_per_token": [0.42491859197616577, 0.5401174426078796, 0.5078622698783875, 0.5671795606613159, 0.5338308215141296, 0.7052080631256104, 0.9965019226074219, 0.9992721676826477, 0.8702996969223022, 0.21793632209300995, 0.9695422053337097, 0.9806720614433289, 0.7883416414260864, 1.0, 0.9180929660797119, 0.9052408337593079, 0.9991720914840698, 1.0, 0.9282515048980713, 0.7349711656570435], "n_positions_probed": 1, "per_restart_best": [2.9096896648406982]}
+{"step": 420, "discrete_loss": 5.360703945159912, "best_sample_loss": 2.928950071334839, "soft_loss": 4.261490821838379, "best_discrete": 2.9096896648406982, "best_soft": 2.477175235748291, "best_argmax": 4.538334369659424, "best_sampling": 2.9096896648406982, "relax_gap": 0.20505014538510263, "n_match": 9, "g_first_norm": 178.61212158203125, "vocab_size": 50257, "entropy": 0.5694665312767029, "entropy_per_token": [0.7400678992271423, 0.7911925315856934, 1.0829946994781494, 1.056970477104187, 0.7202865481376648, 0.657204270362854, 0.023976586759090424, 0.006544132251292467, 0.3828200399875641, 2.8220622539520264, 0.15712541341781616, 0.11079996824264526, 0.5782889127731323, 2.2632187235416268e-09, 0.3088394105434418, 0.530282735824585, 0.007019443437457085, 4.25166096773566e-10, 0.4945758581161499, 0.9182799458503723], "max_p": 0.798503041267395, "max_p_per_token": [0.8369049429893494, 0.561147928237915, 0.5039488077163696, 0.559195339679718, 0.5240994095802307, 0.7054629921913147, 0.996382474899292, 0.9992256164550781, 0.8782482147216797, 0.2347911298274994, 0.967452347278595, 0.9812306761741638, 0.8008742928504944, 1.0, 0.9198879599571228, 0.885230302810669, 0.9991857409477234, 1.0, 0.9160603284835815, 0.700731635093689], "n_positions_probed": 1, "per_restart_best": [2.9096896648406982]}
+{"step": 421, "discrete_loss": 5.212528228759766, "best_sample_loss": 3.023940324783325, "soft_loss": 4.291335582733154, "best_discrete": 2.9096896648406982, "best_soft": 2.477175235748291, "best_argmax": 4.538334369659424, "best_sampling": 2.9096896648406982, "relax_gap": 0.17672664887338055, "n_match": 9, "g_first_norm": 233.01861572265625, "vocab_size": 50257, "entropy": 0.5883473753929138, "entropy_per_token": [0.8124831914901733, 0.8192511796951294, 1.1000410318374634, 1.1004562377929688, 0.7209550738334656, 0.6850835084915161, 0.024021156132221222, 0.007162772119045258, 0.3840809762477875, 2.858001470565796, 0.16236761212348938, 0.10568681359291077, 0.5627199411392212, 2.4905897344495997e-09, 0.3028551936149597, 0.6015738844871521, 0.006923246197402477, 4.201612391341314e-10, 0.5319527983665466, 0.9813308715820312], "max_p": 0.7849366068840027, "max_p_per_token": [0.8159825801849365, 0.49247920513153076, 0.48007166385650635, 0.5099760293960571, 0.5145628452301025, 0.6781593561172485, 0.9963746666908264, 0.9991425275802612, 0.8772522807121277, 0.21304894983768463, 0.966033935546875, 0.982429027557373, 0.8106893301010132, 1.0, 0.9221444725990295, 0.8641664385795593, 0.999201238155365, 1.0, 0.9056522250175476, 0.6713651418685913], "n_positions_probed": 1, "per_restart_best": [2.9096896648406982]}
+{"step": 422, "discrete_loss": 5.212528228759766, "best_sample_loss": 2.9955904483795166, "soft_loss": 4.192666053771973, "best_discrete": 2.9096896648406982, "best_soft": 2.477175235748291, "best_argmax": 4.538334369659424, "best_sampling": 2.9096896648406982, "relax_gap": 0.19565595239576328, "n_match": 9, "g_first_norm": 168.39041137695312, "vocab_size": 50257, "entropy": 0.5964164733886719, "entropy_per_token": [0.8292424082756042, 0.8275357484817505, 1.099076509475708, 1.1096611022949219, 0.7176915407180786, 0.7078567743301392, 0.0247089471668005, 0.007639830466359854, 0.37709370255470276, 2.7953338623046875, 0.16772882640361786, 0.10094459354877472, 0.5573289394378662, 2.7349471576343376e-09, 0.2960852086544037, 0.6848228573799133, 0.00685298815369606, 4.1878253642657626e-10, 0.5779630541801453, 1.0407625436782837], "max_p": 0.7826511263847351, "max_p_per_token": [0.8103771209716797, 0.4975425899028778, 0.48717930912971497, 0.4854893982410431, 0.5244606733322144, 0.6537311673164368, 0.9962521195411682, 0.9990779161453247, 0.8804385662078857, 0.2592807412147522, 0.9645600318908691, 0.9835030436515808, 0.8146217465400696, 1.0, 0.924674391746521, 0.8380048871040344, 0.9992138147354126, 1.0, 0.8916831016540527, 0.6429310441017151], "n_positions_probed": 1, "per_restart_best": [2.9096896648406982]}
+{"step": 423, "discrete_loss": 5.212528228759766, "best_sample_loss": 2.942042589187622, "soft_loss": 4.121038436889648, "best_discrete": 2.9096896648406982, "best_soft": 2.477175235748291, "best_argmax": 4.538334369659424, "best_sampling": 2.9096896648406982, "relax_gap": 0.20939738721181353, "n_match": 9, "g_first_norm": 194.29725646972656, "vocab_size": 50257, "entropy": 0.614082396030426, "entropy_per_token": [0.860582172870636, 0.8355674743652344, 1.0973830223083496, 1.2076278924942017, 0.7142555713653564, 0.7258840203285217, 0.025080587714910507, 0.008362224325537682, 0.37301358580589294, 2.8067498207092285, 0.17256875336170197, 0.09740082174539566, 0.5584481954574585, 2.994733572236896e-09, 0.290105938911438, 0.7740647196769714, 0.0067818136885762215, 4.1778269732617446e-10, 0.6274888515472412, 1.100282073020935], "max_p": 0.7761194109916687, "max_p_per_token": [0.80030757188797, 0.5019468665122986, 0.4924333393573761, 0.44627586007118225, 0.5343429446220398, 0.6325200200080872, 0.9961856007575989, 0.9989789724349976, 0.8821512460708618, 0.25552913546562195, 0.963225781917572, 0.9843148589134216, 0.8149750828742981, 1.0, 0.9268929362297058, 0.8079178929328918, 0.9992262125015259, 1.0, 0.8743945956230164, 0.6107680797576904], "n_positions_probed": 1, "per_restart_best": [2.9096896648406982]}
+{"step": 424, "discrete_loss": 5.360703945159912, "best_sample_loss": 3.0109994411468506, "soft_loss": 4.057520866394043, "best_discrete": 2.9096896648406982, "best_soft": 2.477175235748291, "best_argmax": 4.538334369659424, "best_sampling": 2.9096896648406982, "relax_gap": 0.2430992444457767, "n_match": 9, "g_first_norm": 179.37538146972656, "vocab_size": 50257, "entropy": 0.6112288236618042, "entropy_per_token": [0.878220796585083, 0.8426104187965393, 1.0892776250839233, 1.2032101154327393, 0.43996357917785645, 0.7411304712295532, 0.025631524622440338, 0.009060812182724476, 0.36612698435783386, 2.7739691734313965, 0.17717677354812622, 0.09429791569709778, 0.5680723190307617, 3.2717557552075505e-09, 0.28405916690826416, 0.8697729110717773, 0.0067160711623728275, 4.186743174372509e-10, 0.6972414255142212, 1.1580384969711304], "max_p": 0.7874593138694763, "max_p_per_token": [0.7937712669372559, 0.5091609954833984, 0.506528377532959, 0.44131627678871155, 0.8437089323997498, 0.6131499409675598, 0.996086597442627, 0.9988818764686584, 0.8852623701095581, 0.27742457389831543, 0.9619418382644653, 0.985017716884613, 0.8108106255531311, 1.0, 0.9290958046913147, 0.773162841796875, 0.9992382526397705, 1.0, 0.8475509881973267, 0.577078104019165], "n_positions_probed": 1, "per_restart_best": [2.9096896648406982]}
+{"step": 425, "discrete_loss": 5.360703945159912, "best_sample_loss": 2.9096896648406982, "soft_loss": 4.076268196105957, "best_discrete": 2.9096896648406982, "best_soft": 2.477175235748291, "best_argmax": 4.538334369659424, "best_sampling": 2.9096896648406982, "relax_gap": 0.2396020676004035, "n_match": 9, "g_first_norm": 254.21397399902344, "vocab_size": 50257, "entropy": 0.6393179893493652, "entropy_per_token": [1.0054526329040527, 0.8526724576950073, 1.0990140438079834, 1.2112332582473755, 0.45678287744522095, 0.755998969078064, 0.025424594059586525, 0.010095109231770039, 0.36581987142562866, 2.8722479343414307, 0.1823638379573822, 0.09158466011285782, 0.5790057182312012, 3.5461338310227575e-09, 0.2773042619228363, 0.9728329181671143, 0.006627736613154411, 4.162507005744942e-10, 0.8065561056137085, 1.2153425216674805], "max_p": 0.772779643535614, "max_p_per_token": [0.7531646490097046, 0.4869052469730377, 0.49435994029045105, 0.4321751594543457, 0.8338453769683838, 0.603874683380127, 0.9961239695549011, 0.9987363219261169, 0.885136604309082, 0.2188100814819336, 0.9605107307434082, 0.9856374859809875, 0.8061042428016663, 1.0, 0.9314176440238953, 0.7318055629730225, 0.9992520213127136, 1.0, 0.8002254962921143, 0.5375087857246399], "n_positions_probed": 1, "per_restart_best": [2.9096896648406982]}
+{"step": 426, "discrete_loss": 5.360703945159912, "best_sample_loss": 3.0240094661712646, "soft_loss": 3.941937208175659, "best_discrete": 2.9096896648406982, "best_soft": 2.477175235748291, "best_argmax": 4.538334369659424, "best_sampling": 2.9096896648406982, "relax_gap": 0.26466052807583845, "n_match": 9, "g_first_norm": 161.50439453125, "vocab_size": 50257, "entropy": 0.6566939353942871, "entropy_per_token": [1.0699323415756226, 0.858543872833252, 1.0995069742202759, 1.2158665657043457, 0.47230958938598633, 0.7658807635307312, 0.025834284722805023, 0.01089246105402708, 0.3571699261665344, 2.8044166564941406, 0.18802465498447418, 0.08872225880622864, 0.599822998046875, 3.8310528083229656e-09, 0.2688334584236145, 1.0744566917419434, 0.006557955406606197, 4.1750702894916003e-10, 0.964688777923584, 1.2624180316925049], "max_p": 0.7649111747741699, "max_p_per_token": [0.7319204211235046, 0.489958792924881, 0.4948355257511139, 0.4312100112438202, 0.824327826499939, 0.596919596195221, 0.9960502982139587, 0.9986222982406616, 0.8891236782073975, 0.2644965350627899, 0.9589059352874756, 0.9862603545188904, 0.7961307764053345, 1.0, 0.9342755675315857, 0.6867285966873169, 0.9992638230323792, 1.0, 0.7128982543945312, 0.5062950849533081], "n_positions_probed": 1, "per_restart_best": [2.9096896648406982]}
+{"step": 427, "discrete_loss": 5.360703945159912, "best_sample_loss": 2.9096896648406982, "soft_loss": 3.749807357788086, "best_discrete": 2.9096896648406982, "best_soft": 2.477175235748291, "best_argmax": 4.538334369659424, "best_sampling": 2.9096896648406982, "relax_gap": 0.3005009423857994, "n_match": 9, "g_first_norm": 191.43270874023438, "vocab_size": 50257, "entropy": 0.6783484220504761, "entropy_per_token": [1.137554407119751, 0.8627593517303467, 1.0969914197921753, 1.2196987867355347, 0.490356981754303, 0.7727124691009521, 0.02605205401778221, 0.004504016134887934, 0.3520761728286743, 2.8415355682373047, 0.1921168565750122, 0.0871177613735199, 0.6226215362548828, 4.1036334330613045e-09, 0.26018673181533813, 1.1663795709609985, 0.006480556912720203, 4.172397150004059e-10, 1.127745509147644, 1.3000783920288086], "max_p": 0.7506163716316223, "max_p_per_token": [0.7094264030456543, 0.4956449270248413, 0.4986230134963989, 0.4261205792427063, 0.8127793669700623, 0.5952397584915161, 0.9960110187530518, 0.9995182752609253, 0.8913467526435852, 0.24564379453659058, 0.9577416181564331, 0.9866324067115784, 0.7845143675804138, 1.0, 0.9371435642242432, 0.6413376331329346, 0.9992761015892029, 1.0, 0.5572013258934021, 0.4781267046928406], "n_positions_probed": 1, "per_restart_best": [2.9096896648406982]}
+{"step": 428, "discrete_loss": 4.869917392730713, "best_sample_loss": 2.933802366256714, "soft_loss": 3.387226104736328, "best_discrete": 2.9096896648406982, "best_soft": 2.477175235748291, "best_argmax": 4.538334369659424, "best_sampling": 2.9096896648406982, "relax_gap": 0.3044592276262399, "n_match": 8, "g_first_norm": 169.4162139892578, "vocab_size": 50257, "entropy": 0.6848556399345398, "entropy_per_token": [1.1732290983200073, 0.8678444623947144, 1.0936182737350464, 1.2200456857681274, 0.5159828066825867, 0.7774635553359985, 0.026644494384527206, 0.005292746238410473, 0.35374709963798523, 2.809424877166748, 0.1948079913854599, 0.0870322585105896, 0.6511631011962891, 4.33274438549347e-09, 0.2508070170879364, 1.2442848682403564, 0.006386194843798876, 4.129041830669422e-10, 1.103036642074585, 1.316301941871643], "max_p": 0.7498049139976501, "max_p_per_token": [0.697470486164093, 0.49685508012771606, 0.5043156743049622, 0.42038413882255554, 0.7953099608421326, 0.5954502820968628, 0.9959035515785217, 0.999422550201416, 0.8933637738227844, 0.266032338142395, 0.9569345712661743, 0.9867151379585266, 0.7689937949180603, 1.0, 0.9401817321777344, 0.5978661775588989, 0.9992889165878296, 1.0, 0.6115808486938477, 0.47002917528152466], "n_positions_probed": 1, "per_restart_best": [2.9096896648406982]}
+{"step": 429, "discrete_loss": 5.360703945159912, "best_sample_loss": 4.625268459320068, "soft_loss": 3.0200581550598145, "best_discrete": 2.9096896648406982, "best_soft": 2.477175235748291, "best_argmax": 4.538334369659424, "best_sampling": 2.9096896648406982, "relax_gap": 0.43663030341629416, "n_match": 9, "g_first_norm": 216.65597534179688, "vocab_size": 50257, "entropy": 0.7089118361473083, "entropy_per_token": [1.240835189819336, 0.8662459254264832, 1.0966966152191162, 1.2093067169189453, 0.5624826550483704, 0.7793239951133728, 0.027780063450336456, 0.005239318590611219, 0.3556702136993408, 2.989223003387451, 0.19680848717689514, 0.08991163223981857, 0.6687682867050171, 4.529134844943883e-09, 0.23667970299720764, 1.289815902709961, 0.006271406076848507, 4.0618558516669623e-10, 1.2945575714111328, 1.2626193761825562], "max_p": 0.7365323901176453, "max_p_per_token": [0.6715414524078369, 0.5153051614761353, 0.5059431791305542, 0.4366150498390198, 0.7592234015464783, 0.5954545140266418, 0.9956961870193481, 0.9994304776191711, 0.8924276232719421, 0.17008629441261292, 0.9561911821365356, 0.9863042235374451, 0.7582092881202698, 1.0, 0.9446401000022888, 0.5760433077812195, 0.9993042945861816, 1.0, 0.42991358041763306, 0.5383191704750061], "n_positions_probed": 1, "per_restart_best": [2.9096896648406982]}
+{"step": 430, "discrete_loss": 4.869917392730713, "best_sample_loss": 4.024265289306641, "soft_loss": 3.031715154647827, "best_discrete": 2.9096896648406982, "best_soft": 2.477175235748291, "best_argmax": 4.538334369659424, "best_sampling": 2.9096896648406982, "relax_gap": 0.37746066100151016, "n_match": 8, "g_first_norm": 201.79684448242188, "vocab_size": 50257, "entropy": 0.7124242186546326, "entropy_per_token": [1.2550126314163208, 0.8686126470565796, 1.1049546003341675, 1.2056233882904053, 0.5876049995422363, 0.7808926701545715, 0.02902933396399021, 0.006367517169564962, 0.3502156138420105, 2.874772071838379, 0.4862178862094879, 0.09353295713663101, 0.6923372745513916, 4.697894517846635e-09, 0.2280518114566803, 1.336568832397461, 0.00614901352673769, 4.0057285266570375e-10, 1.111643671989441, 1.230896234512329], "max_p": 0.7462450861930847, "max_p_per_token": [0.6649328470230103, 0.5277097225189209, 0.4979317784309387, 0.4299848973751068, 0.7375543713569641, 0.5968654751777649, 0.9954659938812256, 0.9992896318435669, 0.8952192664146423, 0.24842186272144318, 0.8550805449485779, 0.9857008457183838, 0.7441257238388062, 1.0, 0.9472822546958923, 0.5470480918884277, 0.9993197917938232, 1.0, 0.6945820450782776, 0.5583861470222473], "n_positions_probed": 1, "per_restart_best": [2.9096896648406982]}
+{"step": 431, "discrete_loss": 4.869917392730713, "best_sample_loss": 2.9253592491149902, "soft_loss": 3.052736282348633, "best_discrete": 2.9096896648406982, "best_soft": 2.477175235748291, "best_argmax": 4.538334369659424, "best_sampling": 2.9096896648406982, "relax_gap": 0.3731441344558682, "n_match": 8, "g_first_norm": 270.5791931152344, "vocab_size": 50257, "entropy": 0.7462109923362732, "entropy_per_token": [1.3535486459732056, 0.8867195844650269, 1.104118824005127, 1.1969724893569946, 0.6171373128890991, 0.7833283543586731, 0.02933075651526451, 0.006486848928034306, 0.35350239276885986, 3.0697481632232666, 0.4671435058116913, 0.10109510272741318, 0.6905966997146606, 4.908443429485487e-09, 0.21668094396591187, 1.3364348411560059, 0.0060953604988753796, 4.0977848891898816e-10, 1.5087624788284302, 1.1965175867080688], "max_p": 0.7261061668395996, "max_p_per_token": [0.6225756406784058, 0.49106326699256897, 0.5027024149894714, 0.44253748655319214, 0.7081433534622192, 0.5976911783218384, 0.995410144329071, 0.9992766976356506, 0.8934946656227112, 0.1433374136686325, 0.8631934523582458, 0.9844255447387695, 0.7438942790031433, 1.0, 0.9507731795310974, 0.5548020005226135, 0.99932861328125, 1.0, 0.4596732258796692, 0.5698005557060242], "n_positions_probed": 1, "per_restart_best": [2.9096896648406982]}
+{"step": 432, "discrete_loss": 4.869917392730713, "best_sample_loss": 3.4503746032714844, "soft_loss": 2.7606613636016846, "best_discrete": 2.9096896648406982, "best_soft": 2.477175235748291, "best_argmax": 4.538334369659424, "best_sampling": 2.9096896648406982, "relax_gap": 0.43311946775062304, "n_match": 8, "g_first_norm": 194.75711059570312, "vocab_size": 50257, "entropy": 0.7471535205841064, "entropy_per_token": [1.372421383857727, 0.8867834806442261, 1.1177589893341064, 1.1851834058761597, 0.6333776712417603, 0.7819473147392273, 0.031118689104914665, 0.0072563327848911285, 0.35212963819503784, 2.979274034500122, 0.4511342942714691, 0.11202868819236755, 0.7933639287948608, 4.947944276523231e-09, 0.20825502276420593, 1.368593692779541, 0.006062122993171215, 4.0849346127913577e-10, 1.5122809410095215, 1.1441019773483276], "max_p": 0.7327178716659546, "max_p_per_token": [0.6094672083854675, 0.5092259645462036, 0.48962265253067017, 0.4506768584251404, 0.690264880657196, 0.6014231443405151, 0.9950761198997498, 0.9991788268089294, 0.8944802284240723, 0.21957993507385254, 0.8699895739555359, 0.982441246509552, 0.7236914038658142, 1.0, 0.9532635807991028, 0.5339848399162292, 0.9993334412574768, 1.0, 0.530102550983429, 0.6025540828704834], "n_positions_probed": 1, "per_restart_best": [2.9096896648406982]}
+{"step": 433, "discrete_loss": 4.869917392730713, "best_sample_loss": 4.116742134094238, "soft_loss": 2.657020330429077, "best_discrete": 2.9096896648406982, "best_soft": 2.477175235748291, "best_argmax": 4.538334369659424, "best_sampling": 2.9096896648406982, "relax_gap": 0.4544013550629852, "n_match": 8, "g_first_norm": 171.7504119873047, "vocab_size": 50257, "entropy": 0.7622155547142029, "entropy_per_token": [1.4238770008087158, 0.8908474445343018, 1.1223249435424805, 1.1793016195297241, 0.649072527885437, 0.7756122946739197, 0.032841216772794724, 0.007187659852206707, 0.35043075680732727, 3.037599563598633, 0.42125025391578674, 0.12553492188453674, 0.7870166301727295, 1.5787333040861995e-06, 0.19718284904956818, 1.3643205165863037, 0.006037415005266666, 4.0451678118280654e-10, 1.7515039443969727, 1.1223665475845337], "max_p": 0.7240029573440552, "max_p_per_token": [0.5828335285186768, 0.516234278678894, 0.48680663108825684, 0.4519961476325989, 0.6700146794319153, 0.6148390173912048, 0.9947500824928284, 0.9991890788078308, 0.8952885270118713, 0.19527263939380646, 0.8822205066680908, 0.9799109697341919, 0.7285917401313782, 0.9999998807907104, 0.9565179347991943, 0.5394785404205322, 0.9993383288383484, 1.0, 0.37125757336616516, 0.6155180931091309], "n_positions_probed": 1, "per_restart_best": [2.9096896648406982]}
+{"step": 434, "discrete_loss": 4.869917392730713, "best_sample_loss": 4.086132049560547, "soft_loss": 2.5799450874328613, "best_discrete": 2.9096896648406982, "best_soft": 2.477175235748291, "best_argmax": 4.538334369659424, "best_sampling": 2.9096896648406982, "relax_gap": 0.4702281621277755, "n_match": 8, "g_first_norm": 168.02920532226562, "vocab_size": 50257, "entropy": 0.7618865370750427, "entropy_per_token": [1.454206943511963, 0.8974359035491943, 1.1227145195007324, 1.1760843992233276, 0.6668850183486938, 0.7705647945404053, 0.03481600061058998, 0.008119458332657814, 0.34884896874427795, 2.979391098022461, 0.40993016958236694, 0.1390051394701004, 0.7915313243865967, 1.5216512565530138e-06, 0.20527011156082153, 1.3765121698379517, 0.0060157328844070435, 3.9933642503875433e-10, 1.7504334449768066, 1.0999648571014404], "max_p": 0.7282048463821411, "max_p_per_token": [0.5653262734413147, 0.518147349357605, 0.48815739154815674, 0.44439971446990967, 0.6441465020179749, 0.6239945292472839, 0.9943715333938599, 0.9990687966346741, 0.8963137865066528, 0.23060712218284607, 0.8866811990737915, 0.9772517681121826, 0.7273603081703186, 0.9999998807907104, 0.956516444683075, 0.5321590304374695, 0.9993417859077454, 1.0, 0.45228052139282227, 0.6279726624488831], "n_positions_probed": 1, "per_restart_best": [2.9096896648406982]}
+{"step": 435, "discrete_loss": 4.869917392730713, "best_sample_loss": 2.9096896648406982, "soft_loss": 2.532217502593994, "best_discrete": 2.9096896648406982, "best_soft": 2.477175235748291, "best_argmax": 4.538334369659424, "best_sampling": 2.9096896648406982, "relax_gap": 0.48002865379732823, "n_match": 8, "g_first_norm": 150.17138671875, "vocab_size": 50257, "entropy": 0.7065334320068359, "entropy_per_token": [1.5032986402511597, 0.9041936993598938, 1.1240133047103882, 1.1741939783096313, 0.6659986972808838, 0.764933705329895, 0.03677020221948624, 0.008069825358688831, 0.34741801023483276, 3.0347418785095215, 0.381672203540802, 0.1581362783908844, 0.7868928909301758, 1.4594344293072936e-06, 0.19616039097309113, 0.0017599971033632755, 0.006079080980271101, 3.9402842100244584e-10, 1.938185691833496, 1.0981483459472656], "max_p": 0.7449231147766113, "max_p_per_token": [0.5366849303245544, 0.5196337103843689, 0.48873743414878845, 0.4423108696937561, 0.644618034362793, 0.6346157789230347, 0.9939919114112854, 0.9990766048431396, 0.8970895409584045, 0.20684412121772766, 0.8976091146469116, 0.9732735753059387, 0.7311813235282898, 0.9999998807907104, 0.9590242505073547, 0.9998440742492676, 0.9993358254432678, 1.0, 0.3457392156124115, 0.6288521885871887], "n_positions_probed": 1, "per_restart_best": [2.9096896648406982]}
+{"step": 436, "discrete_loss": 4.869917392730713, "best_sample_loss": 3.2941787242889404, "soft_loss": 2.6358346939086914, "best_discrete": 2.9096896648406982, "best_soft": 2.477175235748291, "best_argmax": 4.538334369659424, "best_sampling": 2.9096896648406982, "relax_gap": 0.4587516622267185, "n_match": 8, "g_first_norm": 145.3824462890625, "vocab_size": 50257, "entropy": 0.7087175250053406, "entropy_per_token": [1.5283687114715576, 0.902271032333374, 1.12467622756958, 1.167676568031311, 0.6692455410957336, 0.7594752311706543, 0.039075855165719986, 0.00820067711174488, 0.34721630811691284, 3.008556365966797, 0.3621136248111725, 0.1777847558259964, 0.7818176746368408, 1.4193425386110903e-06, 0.19183817505836487, 0.002100778743624687, 0.006305827293545008, 3.8568712112940773e-10, 2.019197463989258, 1.0784276723861694], "max_p": 0.7463208436965942, "max_p_per_token": [0.5188093781471252, 0.5357456803321838, 0.49094516038894653, 0.4382933974266052, 0.6383742690086365, 0.6431335806846619, 0.9935378432273865, 0.9990600943565369, 0.8973979949951172, 0.2269609123468399, 0.9048837423324585, 0.968855082988739, 0.7346305847167969, 0.9999998807907104, 0.960252583026886, 0.999810516834259, 0.9993122816085815, 1.0, 0.33486318588256836, 0.6415507197380066], "n_positions_probed": 1, "per_restart_best": [2.9096896648406982]}
+{"step": 437, "discrete_loss": 4.869917392730713, "best_sample_loss": 4.1729326248168945, "soft_loss": 2.598334789276123, "best_discrete": 2.9096896648406982, "best_soft": 2.477175235748291, "best_argmax": 4.538334369659424, "best_sampling": 2.9096896648406982, "relax_gap": 0.466451978599342, "n_match": 8, "g_first_norm": 133.69891357421875, "vocab_size": 50257, "entropy": 0.7148600816726685, "entropy_per_token": [1.5566871166229248, 0.9085133075714111, 1.1238974332809448, 1.1639224290847778, 0.6718525886535645, 0.7540390491485596, 0.04136822372674942, 0.008158298209309578, 0.3470960557460785, 3.0377540588378906, 0.3457115888595581, 0.20203644037246704, 0.7764198780059814, 1.3791067203783314e-06, 0.18793603777885437, 0.002508982317522168, 0.006457547657191753, 3.766867651133765e-10, 2.0995712280273438, 1.0632688999176025], "max_p": 0.7442696690559387, "max_p_per_token": [0.4990961253643036, 0.53473961353302, 0.49467793107032776, 0.4324115216732025, 0.6327307820320129, 0.6514691114425659, 0.9930800795555115, 0.9990662932395935, 0.8976128697395325, 0.21500763297080994, 0.9108092784881592, 0.9629915952682495, 0.73816978931427, 0.9999998807907104, 0.961353600025177, 0.9997691512107849, 0.9992960691452026, 1.0, 0.31350669264793396, 0.6496052145957947], "n_positions_probed": 1, "per_restart_best": [2.9096896648406982]}
+{"step": 438, "discrete_loss": 4.869917392730713, "best_sample_loss": 3.3453667163848877, "soft_loss": 2.5663716793060303, "best_discrete": 2.9096896648406982, "best_soft": 2.477175235748291, "best_argmax": 4.538334369659424, "best_sampling": 2.9096896648406982, "relax_gap": 0.473015356864814, "n_match": 8, "g_first_norm": 137.50791931152344, "vocab_size": 50257, "entropy": 0.7210627794265747, "entropy_per_token": [1.5810152292251587, 0.9109092354774475, 1.1255601644515991, 1.1586036682128906, 0.6730058193206787, 0.7481765747070312, 0.04379371181130409, 0.00810357928276062, 0.34718871116638184, 3.0375618934631348, 0.3312976658344269, 0.2315180003643036, 0.7701675891876221, 1.3398685041465797e-06, 0.18413326144218445, 0.0030054496601223946, 0.006629578769207001, 3.671328796528428e-10, 2.2121236324310303, 1.0484604835510254], "max_p": 0.7410568594932556, "max_p_per_token": [0.4798731505870819, 0.5392173528671265, 0.49559667706489563, 0.4300593137741089, 0.629388689994812, 0.6599689722061157, 0.9925888776779175, 0.99907386302948, 0.8977339267730713, 0.21969759464263916, 0.9158749580383301, 0.9552914500236511, 0.7421773672103882, 0.9999998807907104, 0.9624180793762207, 0.9997177720069885, 0.9992780089378357, 1.0, 0.2457364797592163, 0.657444179058075], "n_positions_probed": 1, "per_restart_best": [2.9096896648406982]}
+{"step": 439, "discrete_loss": 4.869917392730713, "best_sample_loss": 4.131067276000977, "soft_loss": 2.566640853881836, "best_discrete": 2.9096896648406982, "best_soft": 2.477175235748291, "best_argmax": 4.538334369659424, "best_sampling": 2.9096896648406982, "relax_gap": 0.47296008394042977, "n_match": 8, "g_first_norm": 144.7877960205078, "vocab_size": 50257, "entropy": 0.7332674860954285, "entropy_per_token": [1.5990574359893799, 0.9103273153305054, 1.1199584007263184, 1.152031421661377, 0.6791326403617859, 0.742787778377533, 0.04588547348976135, 0.00804317370057106, 0.34414130449295044, 3.0884594917297363, 0.31125447154045105, 0.2670060992240906, 0.7612156867980957, 1.311801497649867e-06, 0.18176105618476868, 0.0036217886954545975, 0.006769349332898855, 3.560277350711516e-10, 2.2337207794189453, 1.2101740837097168], "max_p": 0.7303314208984375, "max_p_per_token": [0.46505510807037354, 0.5430415868759155, 0.5050438642501831, 0.4292289614677429, 0.6163298487663269, 0.6675575971603394, 0.9921597838401794, 0.999082088470459, 0.8992088437080383, 0.1936791092157364, 0.9227129817008972, 0.9452687501907349, 0.7476766109466553, 0.9999998807907104, 0.9630783796310425, 0.999652624130249, 0.9992632269859314, 1.0, 0.2781361937522888, 0.44045257568359375], "n_positions_probed": 1, "per_restart_best": [2.9096896648406982]}
+{"step": 440, "discrete_loss": 4.828214168548584, "best_sample_loss": 2.9772186279296875, "soft_loss": 2.5881004333496094, "best_discrete": 2.9096896648406982, "best_soft": 2.477175235748291, "best_argmax": 4.538334369659424, "best_sampling": 2.9096896648406982, "relax_gap": 0.46396320813423614, "n_match": 8, "g_first_norm": 172.90028381347656, "vocab_size": 50257, "entropy": 0.6815668940544128, "entropy_per_token": [0.553741991519928, 0.9031798839569092, 1.123323678970337, 1.140528917312622, 0.6692550182342529, 0.7344459295272827, 0.04815905913710594, 0.008536380715668201, 0.34272855520248413, 3.0213065147399902, 0.29267755150794983, 0.30463865399360657, 0.7670199871063232, 1.2547527603601338e-06, 0.1746591329574585, 0.004368743859231472, 0.006881778594106436, 3.489220024022188e-10, 2.337085723876953, 1.1987988948822021], "max_p": 0.7569426894187927, "max_p_per_token": [0.8755847215652466, 0.5603294968605042, 0.5046008825302124, 0.4415242671966553, 0.6330034136772156, 0.6779207587242126, 0.9916877150535583, 0.999018669128418, 0.8999070525169373, 0.24089600145816803, 0.9288329482078552, 0.9337000846862793, 0.7442221641540527, 0.9999998807907104, 0.9649748206138611, 0.9995713829994202, 0.9992524981498718, 1.0, 0.2530956268310547, 0.4907319247722626], "n_positions_probed": 1, "per_restart_best": [2.9096896648406982]}
+{"step": 441, "discrete_loss": 4.828214168548584, "best_sample_loss": 3.058880567550659, "soft_loss": 2.7457404136657715, "best_discrete": 2.9096896648406982, "best_soft": 2.477175235748291, "best_argmax": 4.538334369659424, "best_sampling": 2.9096896648406982, "relax_gap": 0.43131345921815806, "n_match": 8, "g_first_norm": 154.09532165527344, "vocab_size": 50257, "entropy": 0.7007200121879578, "entropy_per_token": [0.6601769328117371, 0.9635775685310364, 1.16111421585083, 1.145611047744751, 0.6928912997245789, 0.7370720505714417, 0.050191860646009445, 0.008282959461212158, 0.34913721680641174, 3.126009225845337, 0.2973061501979828, 0.36671391129493713, 0.7841289043426514, 1.1990681514362223e-06, 0.17694520950317383, 0.005282208323478699, 0.007018791977316141, 3.2703820207480305e-10, 2.31112003326416, 1.1718196868896484], "max_p": 0.7461704611778259, "max_p_per_token": [0.8436543345451355, 0.5051239132881165, 0.45930182933807373, 0.4296298623085022, 0.5806989073753357, 0.6760696768760681, 0.9912609457969666, 0.9990519881248474, 0.897344708442688, 0.18201911449432373, 0.9270364046096802, 0.9129027724266052, 0.7341015934944153, 0.9999998807907104, 0.9644922018051147, 0.9994694590568542, 0.9992376565933228, 1.0, 0.294255793094635, 0.5277575850486755], "n_positions_probed": 1, "per_restart_best": [2.9096896648406982]}
+{"step": 442, "discrete_loss": 4.828214168548584, "best_sample_loss": 2.9701919555664062, "soft_loss": 2.6493282318115234, "best_discrete": 2.9096896648406982, "best_soft": 2.477175235748291, "best_argmax": 4.538334369659424, "best_sampling": 2.9096896648406982, "relax_gap": 0.45128195657320197, "n_match": 8, "g_first_norm": 187.52713012695312, "vocab_size": 50257, "entropy": 0.707979142665863, "entropy_per_token": [0.7287284135818481, 0.9877914786338806, 1.12155020236969, 1.1346321105957031, 0.6959424614906311, 0.7251286506652832, 0.053411953151226044, 0.008488805964589119, 0.35745689272880554, 3.076822280883789, 0.30169880390167236, 0.44362980127334595, 0.7955770492553711, 1.1559042150111054e-06, 0.1781691610813141, 0.00642210990190506, 0.007251071743667126, 3.0847316367932365e-10, 2.3898301124572754, 1.1470508575439453], "max_p": 0.7466446757316589, "max_p_per_token": [0.8203155398368835, 0.4800601899623871, 0.5063285827636719, 0.44539591670036316, 0.5711144208908081, 0.6897632479667664, 0.990575909614563, 0.9990259408950806, 0.8941702246665955, 0.21787312626838684, 0.925313413143158, 0.8831512928009033, 0.7273508310317993, 0.9999998807907104, 0.9642733335494995, 0.999338686466217, 0.9992116689682007, 1.0, 0.2671002745628357, 0.5525302290916443], "n_positions_probed": 1, "per_restart_best": [2.9096896648406982]}
+{"step": 443, "discrete_loss": 4.703911304473877, "best_sample_loss": 2.9756813049316406, "soft_loss": 2.577000617980957, "best_discrete": 2.9096896648406982, "best_soft": 2.477175235748291, "best_argmax": 4.538334369659424, "best_sampling": 2.9096896648406982, "relax_gap": 0.4521579062237039, "n_match": 8, "g_first_norm": 153.7385711669922, "vocab_size": 50257, "entropy": 0.7262421250343323, "entropy_per_token": [0.8401268124580383, 1.0052330493927002, 1.1899806261062622, 1.1305112838745117, 0.7002283334732056, 0.7235208749771118, 0.056033555418252945, 0.008576781488955021, 0.3642665147781372, 3.1214728355407715, 0.30239707231521606, 0.5320720076560974, 0.8034976720809937, 1.1205262353541912e-06, 0.18054422736167908, 0.007872705347836018, 0.007521865889430046, 2.919540442736235e-10, 2.4200901985168457, 1.1308940649032593], "max_p": 0.7348785996437073, "max_p_per_token": [0.7815800309181213, 0.45197343826293945, 0.41205263137817383, 0.4471527934074402, 0.5551015734672546, 0.6926953196525574, 0.9900098443031311, 0.9990149736404419, 0.8915907144546509, 0.19270306825637817, 0.9248608350753784, 0.8422003984451294, 0.7233231067657471, 0.9999998807907104, 0.9637659192085266, 0.9991674423217773, 0.9991819262504578, 1.0, 0.2649107277393341, 0.5662875175476074], "n_positions_probed": 1, "per_restart_best": [2.9096896648406982]}
+{"step": 444, "discrete_loss": 4.703911304473877, "best_sample_loss": 2.977245807647705, "soft_loss": 2.47996187210083, "best_discrete": 2.9096896648406982, "best_soft": 2.477175235748291, "best_argmax": 4.538334369659424, "best_sampling": 2.9096896648406982, "relax_gap": 0.47278728029115996, "n_match": 8, "g_first_norm": 150.56027221679688, "vocab_size": 50257, "entropy": 0.7355181574821472, "entropy_per_token": [0.9236366152763367, 1.0116872787475586, 1.1555113792419434, 1.1225178241729736, 0.702630877494812, 0.7200047373771667, 0.058944206684827805, 0.008790500462055206, 0.3707636594772339, 3.110501766204834, 0.30962565541267395, 0.6155201196670532, 0.8131149411201477, 9.701998351374641e-07, 0.18302518129348755, 0.0095590241253376, 0.007786833215504885, 2.771065876761014e-10, 2.4766650199890137, 1.110076904296875], "max_p": 0.7370414137840271, "max_p_per_token": [0.7490508556365967, 0.5165187120437622, 0.46006685495376587, 0.4603274166584015, 0.5443257689476013, 0.6975325345993042, 0.9893732666969299, 0.99898761510849, 0.8890656232833862, 0.20332391560077667, 0.9221893548965454, 0.7945014238357544, 0.718389093875885, 1.0, 0.9632192254066467, 0.9989618062973022, 0.9991528987884521, 1.0, 0.2501447796821594, 0.5856971740722656], "n_positions_probed": 1, "per_restart_best": [2.9096896648406982]}
+{"step": 445, "discrete_loss": 4.703911304473877, "best_sample_loss": 2.9708259105682373, "soft_loss": 2.361158847808838, "best_discrete": 2.9096896648406982, "best_soft": 2.361158847808838, "best_argmax": 4.538334369659424, "best_sampling": 2.9096896648406982, "relax_gap": 0.498043501465866, "n_match": 8, "g_first_norm": 132.92172241210938, "vocab_size": 50257, "entropy": 0.7466339468955994, "entropy_per_token": [0.9709039330482483, 1.017173171043396, 1.161805510520935, 1.1155674457550049, 0.7050607204437256, 0.7107400298118591, 0.06160943955183029, 0.008750529028475285, 0.37520867586135864, 3.139392852783203, 0.3123553395271301, 0.688312828540802, 0.817963182926178, 9.509282108410844e-07, 0.18564477562904358, 0.01158512756228447, 0.008081318810582161, 2.636111884335435e-10, 2.539679765701294, 1.1028414964675903], "max_p": 0.7318560481071472, "max_p_per_token": [0.7304669618606567, 0.5214847326278687, 0.4484993815422058, 0.4677623212337494, 0.528710126876831, 0.7069210410118103, 0.9887824654579163, 0.9989932179450989, 0.8874503970146179, 0.19238431751728058, 0.9211496710777283, 0.7412177324295044, 0.7175946831703186, 1.0, 0.9626546502113342, 0.9987075328826904, 0.999121367931366, 1.0, 0.2281162440776825, 0.5971030592918396], "n_positions_probed": 1, "per_restart_best": [2.9096896648406982]}
+{"step": 446, "discrete_loss": 4.703911304473877, "best_sample_loss": 2.906998872756958, "soft_loss": 2.2925124168395996, "best_discrete": 2.906998872756958, "best_soft": 2.2925124168395996, "best_argmax": 4.538334369659424, "best_sampling": 2.906998872756958, "relax_gap": 0.512636980493404, "n_match": 7, "g_first_norm": 131.35879516601562, "vocab_size": 50257, "entropy": 0.7532884478569031, "entropy_per_token": [1.0107461214065552, 1.0248794555664062, 1.157003402709961, 1.1095608472824097, 0.7059062123298645, 0.7028689384460449, 0.06418517976999283, 0.008765709586441517, 0.37830787897109985, 3.1330084800720215, 0.3108808398246765, 0.7569305896759033, 0.8231962323188782, 9.396959512741887e-07, 0.19062916934490204, 0.013972867280244827, 0.008403541520237923, 2.490874451144265e-10, 2.563290596008301, 1.1032320261001587], "max_p": 0.7288640141487122, "max_p_per_token": [0.7133470177650452, 0.520192563533783, 0.45473888516426086, 0.47298118472099304, 0.5199257731437683, 0.7146735787391663, 0.988204836845398, 0.9989914298057556, 0.8864693641662598, 0.2011810541152954, 0.9216356873512268, 0.6773027777671814, 0.7182851433753967, 1.0, 0.9614881277084351, 0.99839848279953, 0.9990870952606201, 1.0, 0.22713404893875122, 0.6032429337501526], "n_positions_probed": 1, "per_restart_best": [2.906998872756958]}
+{"step": 447, "discrete_loss": 4.703911304473877, "best_sample_loss": 2.9906609058380127, "soft_loss": 2.206181764602661, "best_discrete": 2.906998872756958, "best_soft": 2.206181764602661, "best_argmax": 4.538334369659424, "best_sampling": 2.906998872756958, "relax_gap": 0.530989931186762, "n_match": 7, "g_first_norm": 125.82911682128906, "vocab_size": 50257, "entropy": 0.7627413868904114, "entropy_per_token": [1.0450719594955444, 1.0400934219360352, 1.1648859977722168, 1.109006643295288, 0.7055091857910156, 0.6970210075378418, 0.066575787961483, 0.008885622024536133, 0.3804829716682434, 3.141225576400757, 0.3070431649684906, 0.8342878818511963, 0.8300775289535522, 9.347521654490265e-07, 0.1977221816778183, 0.016777219250798225, 0.008762244135141373, 2.3433044393783575e-10, 2.5886423587799072, 1.1127549409866333], "max_p": 0.7211896777153015, "max_p_per_token": [0.6976178884506226, 0.5028819441795349, 0.4446977376937866, 0.47228819131851196, 0.5218507051467896, 0.7203335762023926, 0.987662672996521, 0.9989769458770752, 0.8859203457832336, 0.19640882313251495, 0.923077404499054, 0.5752549171447754, 0.7196905016899109, 1.0, 0.9597765207290649, 0.9980236291885376, 0.9990481734275818, 1.0, 0.21790458261966705, 0.602378785610199], "n_positions_probed": 1, "per_restart_best": [2.906998872756958]}
+{"step": 448, "discrete_loss": 4.703911304473877, "best_sample_loss": 3.0075843334198, "soft_loss": 2.123553991317749, "best_discrete": 2.906998872756958, "best_soft": 2.123553991317749, "best_argmax": 4.538334369659424, "best_sampling": 2.906998872756958, "relax_gap": 0.5485556903893909, "n_match": 7, "g_first_norm": 131.53958129882812, "vocab_size": 50257, "entropy": 0.7827498316764832, "entropy_per_token": [1.0704184770584106, 1.0579978227615356, 1.160093903541565, 1.1115822792053223, 0.705256462097168, 0.6932841539382935, 0.06903627514839172, 0.009373943321406841, 0.716127872467041, 3.1350390911102295, 0.31944289803504944, 0.843747615814209, 0.8416188955307007, 9.357964358969184e-07, 0.2057705670595169, 0.020018182694911957, 0.009099374525249004, 2.1977827890928836e-10, 2.5581400394439697, 1.1289470195770264], "max_p": 0.7031404376029968, "max_p_per_token": [0.6853728294372559, 0.47860485315322876, 0.45655331015586853, 0.4651612341403961, 0.5220093727111816, 0.7242773771286011, 0.9870989918708801, 0.9989128112792969, 0.5415809750556946, 0.2006571888923645, 0.9188878536224365, 0.5736185312271118, 0.7213432192802429, 1.0, 0.9577978253364563, 0.9975759387016296, 0.9990116357803345, 1.0, 0.23765607178211212, 0.5966880321502686], "n_positions_probed": 1, "per_restart_best": [2.906998872756958]}
+{"step": 449, "discrete_loss": 4.6477370262146, "best_sample_loss": 4.521013259887695, "soft_loss": 2.5496327877044678, "best_discrete": 2.906998872756958, "best_soft": 2.123553991317749, "best_argmax": 4.538334369659424, "best_sampling": 2.906998872756958, "relax_gap": 0.4514249034909266, "n_match": 7, "g_first_norm": 368.5428161621094, "vocab_size": 50257, "entropy": 0.7739053964614868, "entropy_per_token": [1.1696646213531494, 1.077382206916809, 1.133553147315979, 1.1200064420700073, 0.7053383588790894, 0.7154384255409241, 0.06979362666606903, 0.011379361152648926, 0.6843432188034058, 2.6697628498077393, 0.34783029556274414, 0.8671079874038696, 0.8438594341278076, 9.292070330957358e-07, 0.21456646919250488, 0.024040859192609787, 0.009515265934169292, 2.0721196980488799e-10, 2.661790370941162, 1.152733564376831], "max_p": 0.7069935202598572, "max_p_per_token": [0.6374320983886719, 0.48013314604759216, 0.4882860481739044, 0.45267972350120544, 0.5252248644828796, 0.7087750434875488, 0.9869230389595032, 0.998646080493927, 0.5941673517227173, 0.358599454164505, 0.908231794834137, 0.5537511706352234, 0.7259926199913025, 1.0, 0.9555869102478027, 0.9970017075538635, 0.9989643096923828, 1.0, 0.1862991750240326, 0.5831759572029114], "n_positions_probed": 1, "per_restart_best": [2.906998872756958]}
+{"step": 450, "discrete_loss": 4.804042339324951, "best_sample_loss": 4.403876304626465, "soft_loss": 2.750821590423584, "best_discrete": 2.906998872756958, "best_soft": 2.123553991317749, "best_argmax": 4.538334369659424, "best_sampling": 2.906998872756958, "relax_gap": 0.4273943907808854, "n_match": 6, "g_first_norm": 159.2873077392578, "vocab_size": 50257, "entropy": 0.8250799179077148, "entropy_per_token": [1.156745433807373, 1.0922858715057373, 1.1802092790603638, 1.120698094367981, 0.7061849236488342, 0.6964501142501831, 0.06984938681125641, 0.012101240456104279, 0.6810652613639832, 3.120710849761963, 0.8050051331520081, 0.8711241483688354, 0.8671766519546509, 9.395727147420985e-07, 0.2252354770898819, 0.028810866177082062, 0.009985478594899178, 1.9858666100436295e-10, 2.6961231231689453, 1.1618375778198242], "max_p": 0.6749369502067566, "max_p_per_token": [0.6428784132003784, 0.41898995637893677, 0.42706429958343506, 0.44406989216804504, 0.5154988169670105, 0.7265720367431641, 0.9869100451469421, 0.9985476136207581, 0.5979474782943726, 0.14531442523002625, 0.5843497514724731, 0.5797916054725647, 0.7193288803100586, 1.0, 0.9528784155845642, 0.9962981343269348, 0.9989116191864014, 1.0, 0.1842227578163147, 0.5791653394699097], "n_positions_probed": 1, "per_restart_best": [2.906998872756958]}
+{"step": 451, "discrete_loss": 4.834355354309082, "best_sample_loss": 3.014286756515503, "soft_loss": 2.6803154945373535, "best_discrete": 2.906998872756958, "best_soft": 2.123553991317749, "best_argmax": 4.538334369659424, "best_sampling": 2.906998872756958, "relax_gap": 0.4455692024897868, "n_match": 6, "g_first_norm": 218.40689086914062, "vocab_size": 50257, "entropy": 0.8239227533340454, "entropy_per_token": [1.2253555059432983, 1.0902646780014038, 1.1236082315444946, 1.0922269821166992, 0.7062513828277588, 0.7045257091522217, 0.07030216604471207, 0.01224430650472641, 0.6631612181663513, 3.1599960327148438, 0.7817516326904297, 0.925073504447937, 0.8974218368530273, 9.161857974504528e-07, 0.2410525679588318, 0.03564140945672989, 0.010662626475095749, 1.8818339941883977e-10, 2.603672504425049, 1.1352427005767822], "max_p": 0.6802984476089478, "max_p_per_token": [0.6036403775215149, 0.4812098443508148, 0.49768003821372986, 0.48140278458595276, 0.5198416709899902, 0.7228969931602478, 0.9868056774139404, 0.998528003692627, 0.6320935487747192, 0.1278558075428009, 0.5574950575828552, 0.5139641761779785, 0.7034295201301575, 1.0, 0.948578953742981, 0.9952448010444641, 0.9988343119621277, 1.0, 0.2389981895685196, 0.5974686145782471], "n_positions_probed": 1, "per_restart_best": [2.906998872756958]}
+{"step": 452, "discrete_loss": 4.977395534515381, "best_sample_loss": 3.5696816444396973, "soft_loss": 2.4492902755737305, "best_discrete": 2.906998872756958, "best_soft": 2.123553991317749, "best_argmax": 4.538334369659424, "best_sampling": 2.906998872756958, "relax_gap": 0.5079172915655772, "n_match": 7, "g_first_norm": 198.55191040039062, "vocab_size": 50257, "entropy": 0.8429505228996277, "entropy_per_token": [1.2164350748062134, 1.1038973331451416, 1.160718321800232, 1.0887348651885986, 0.7064746618270874, 0.6912028193473816, 0.0731840431690216, 0.013134599663317204, 0.6118901968002319, 3.1583991050720215, 0.775505781173706, 0.9437993764877319, 1.1879990100860596, 9.237836025022261e-07, 0.25560110807418823, 0.04351171851158142, 0.011446960270404816, 1.8023672831990467e-10, 2.6767702102661133, 1.1403048038482666], "max_p": 0.6643388867378235, "max_p_per_token": [0.6062350869178772, 0.4608430564403534, 0.44497933983802795, 0.4752882122993469, 0.5164037346839905, 0.7358537912368774, 0.9861343502998352, 0.9984046816825867, 0.7024003267288208, 0.15443329513072968, 0.4942450523376465, 0.5293469429016113, 0.4541119337081909, 1.0, 0.9445746541023254, 0.99397873878479, 0.9987433552742004, 1.0, 0.19343416392803192, 0.5973666310310364], "n_positions_probed": 1, "per_restart_best": [2.906998872756958]}
+{"step": 453, "discrete_loss": 4.977395534515381, "best_sample_loss": 4.190873622894287, "soft_loss": 2.306593894958496, "best_discrete": 2.906998872756958, "best_soft": 2.123553991317749, "best_argmax": 4.538334369659424, "best_sampling": 2.906998872756958, "relax_gap": 0.5365861766533137, "n_match": 7, "g_first_norm": 258.8970031738281, "vocab_size": 50257, "entropy": 0.8228946924209595, "entropy_per_token": [1.177414894104004, 1.109573245048523, 1.1419930458068848, 1.0757321119308472, 0.7064055800437927, 0.6699723601341248, 0.07618696242570877, 0.01317240484058857, 0.4729790687561035, 3.089749574661255, 0.7487916946411133, 0.9821303486824036, 1.1701984405517578, 0.0001291928201681003, 0.2739487886428833, 0.05431542918086052, 0.012254328466951847, 1.7186359280163543e-10, 2.580171823501587, 1.1027734279632568], "max_p": 0.677738606929779, "max_p_per_token": [0.6207551956176758, 0.456735759973526, 0.4711126387119293, 0.4831199049949646, 0.5068246126174927, 0.7525261640548706, 0.985426664352417, 0.9983988404273987, 0.819877028465271, 0.20237450301647186, 0.5760707855224609, 0.4678986072540283, 0.42897045612335205, 0.9999896287918091, 0.9393263459205627, 0.9921464323997498, 0.9986489415168762, 1.0, 0.23860013484954834, 0.6159701943397522], "n_positions_probed": 1, "per_restart_best": [2.906998872756958]}
+{"step": 454, "discrete_loss": 4.703911304473877, "best_sample_loss": 3.8367772102355957, "soft_loss": 2.017913341522217, "best_discrete": 2.906998872756958, "best_soft": 2.017913341522217, "best_argmax": 4.538334369659424, "best_sampling": 2.906998872756958, "relax_gap": 0.5710137349734922, "n_match": 7, "g_first_norm": 168.96022033691406, "vocab_size": 50257, "entropy": 0.8189795613288879, "entropy_per_token": [1.111523985862732, 1.1264832019805908, 1.1757146120071411, 1.0739344358444214, 0.7048990726470947, 0.6557764410972595, 0.07896731048822403, 0.012834815308451653, 0.4207812547683716, 3.0247859954833984, 0.725704550743103, 1.0156748294830322, 1.1675755977630615, 0.00012923390022478998, 0.28727734088897705, 0.06612136960029602, 0.012955720536410809, 1.6450009410196031e-10, 2.591282844543457, 1.1271693706512451], "max_p": 0.6804972887039185, "max_p_per_token": [0.6534489989280701, 0.4116245210170746, 0.4208567142486572, 0.4746706783771515, 0.521598219871521, 0.7623699307441711, 0.9847638010978699, 0.9984446167945862, 0.8517813682556152, 0.24367080628871918, 0.6204501986503601, 0.47310158610343933, 0.44257017970085144, 0.9999896287918091, 0.9354725480079651, 0.990044891834259, 0.9985663294792175, 1.0, 0.2197706699371338, 0.6067492961883545], "n_positions_probed": 1, "per_restart_best": [2.906998872756958]}
+{"step": 455, "discrete_loss": 3.9320104122161865, "best_sample_loss": 4.170648097991943, "soft_loss": 1.9392532110214233, "best_discrete": 2.906998872756958, "best_soft": 1.9392532110214233, "best_argmax": 3.9320104122161865, "best_sampling": 2.906998872756958, "relax_gap": 0.5068036429922859, "n_match": 7, "g_first_norm": 156.42495727539062, "vocab_size": 50257, "entropy": 0.8219796419143677, "entropy_per_token": [1.1249223947525024, 1.1313012838363647, 1.1629374027252197, 1.0775816440582275, 0.7048234939575195, 0.6569433212280273, 0.08070894330739975, 0.012570802122354507, 0.38829970359802246, 3.0655159950256348, 0.7177258729934692, 1.0460325479507446, 1.1660736799240112, 0.0001272784429602325, 0.3003544211387634, 0.11341434717178345, 0.013579688966274261, 1.556119955115065e-10, 2.5456385612487793, 1.1310397386550903], "max_p": 0.6819030046463013, "max_p_per_token": [0.6466134190559387, 0.4186924993991852, 0.44833487272262573, 0.46337610483169556, 0.5170662999153137, 0.7614840269088745, 0.9843448996543884, 0.998480498790741, 0.86955326795578, 0.21355509757995605, 0.6360107064247131, 0.4653474986553192, 0.45837971568107605, 0.9999898672103882, 0.9316230416297913, 0.9823938608169556, 0.9984920024871826, 1.0, 0.2377530336380005, 0.6065689921379089], "n_positions_probed": 1, "per_restart_best": [2.906998872756958]}
+{"step": 456, "discrete_loss": 3.9320104122161865, "best_sample_loss": 4.865699291229248, "soft_loss": 1.8957287073135376, "best_discrete": 2.906998872756958, "best_soft": 1.8957287073135376, "best_argmax": 3.9320104122161865, "best_sampling": 2.906998872756958, "relax_gap": 0.5178729177766714, "n_match": 7, "g_first_norm": 142.47364807128906, "vocab_size": 50257, "entropy": 0.8221141695976257, "entropy_per_token": [1.0956734418869019, 1.1404469013214111, 1.174675464630127, 1.0738458633422852, 0.7039637565612793, 0.6478412747383118, 0.08305889368057251, 0.012266119942069054, 0.3694807291030884, 3.0307040214538574, 0.7063322067260742, 1.0795315504074097, 1.1656631231307983, 0.00012643003719858825, 0.31247827410697937, 0.13597247004508972, 0.014343062415719032, 1.4814521831496563e-10, 2.5528411865234375, 1.1430373191833496], "max_p": 0.6826192736625671, "max_p_per_token": [0.6610288023948669, 0.39373305439949036, 0.43377014994621277, 0.4616386592388153, 0.5221801996231079, 0.7676348090171814, 0.9837754368782043, 0.998521625995636, 0.879228949546814, 0.23718443512916565, 0.6533129215240479, 0.449751079082489, 0.47445252537727356, 0.9999898672103882, 0.928004264831543, 0.9780340790748596, 0.9984002709388733, 1.0, 0.22899192571640015, 0.6027533411979675], "n_positions_probed": 1, "per_restart_best": [2.906998872756958]}
+{"step": 457, "discrete_loss": 3.9320104122161865, "best_sample_loss": 3.964141845703125, "soft_loss": 1.8656575679779053, "best_discrete": 2.906998872756958, "best_soft": 1.8656575679779053, "best_argmax": 3.9320104122161865, "best_sampling": 2.906998872756958, "relax_gap": 0.5255206949143426, "n_match": 7, "g_first_norm": 142.8612060546875, "vocab_size": 50257, "entropy": 0.8262006640434265, "entropy_per_token": [1.1036568880081177, 1.1437615156173706, 1.1677957773208618, 1.0744800567626953, 0.7038660049438477, 0.645939290523529, 0.08498979359865189, 0.011961286887526512, 0.3552236557006836, 3.060145616531372, 0.7048709392547607, 1.1112282276153564, 1.1686217784881592, 0.00012504527694545686, 0.32519084215164185, 0.1627785563468933, 0.015091313049197197, 4.423028054922895e-10, 2.536473274230957, 1.1478134393692017], "max_p": 0.6827613711357117, "max_p_per_token": [0.6573699116706848, 0.39104989171028137, 0.45021000504493713, 0.4556502103805542, 0.5181509256362915, 0.7688258290290833, 0.9833037853240967, 0.9985628724098206, 0.8862810730934143, 0.2164124697446823, 0.6578897833824158, 0.4570547938346863, 0.4814459979534149, 0.9999899864196777, 0.924149215221405, 0.9725767970085144, 0.9983093738555908, 1.0, 0.23612532019615173, 0.6018690466880798], "n_positions_probed": 1, "per_restart_best": [2.906998872756958]}
+{"step": 458, "discrete_loss": 3.9320104122161865, "best_sample_loss": 3.0994045734405518, "soft_loss": 1.8397210836410522, "best_discrete": 2.906998872756958, "best_soft": 1.8397210836410522, "best_argmax": 3.9320104122161865, "best_sampling": 2.906998872756958, "relax_gap": 0.5321169349080802, "n_match": 7, "g_first_norm": 136.6459197998047, "vocab_size": 50257, "entropy": 0.8310562372207642, "entropy_per_token": [1.0875372886657715, 1.1480798721313477, 1.1803241968154907, 1.0716627836227417, 0.7033432126045227, 0.6390519738197327, 0.08729476481676102, 0.01164393499493599, 0.34274131059646606, 3.0357823371887207, 0.7000257968902588, 1.1452089548110962, 1.173349380493164, 0.0001241748541360721, 0.3375014662742615, 0.19439633190631866, 0.015943966805934906, 4.2443368264422077e-10, 2.5917866230010986, 1.1553276777267456], "max_p": 0.6817375421524048, "max_p_per_token": [0.6655166745185852, 0.3732247054576874, 0.4350380003452301, 0.4549732208251953, 0.5195056200027466, 0.773360550403595, 0.982736349105835, 0.9986054301261902, 0.8922674655914307, 0.2326945811510086, 0.6657998561859131, 0.4430197477340698, 0.48636800050735474, 0.9999901056289673, 0.9203614592552185, 0.9657802581787109, 0.9982045888900757, 1.0, 0.22764620184898376, 0.5996575951576233], "n_positions_probed": 1, "per_restart_best": [2.906998872756958]}
+{"step": 459, "discrete_loss": 3.9320104122161865, "best_sample_loss": 3.7813961505889893, "soft_loss": 1.8184562921524048, "best_discrete": 2.906998872756958, "best_soft": 1.8184562921524048, "best_argmax": 3.9320104122161865, "best_sampling": 2.906998872756958, "relax_gap": 0.5375250567743349, "n_match": 7, "g_first_norm": 140.19630432128906, "vocab_size": 50257, "entropy": 0.8445569276809692, "entropy_per_token": [1.0925171375274658, 1.1482044458389282, 1.1747474670410156, 1.0709199905395508, 0.7031398415565491, 0.6360739469528198, 0.08934164047241211, 0.011255311779677868, 0.3321799337863922, 3.0555217266082764, 0.7005232572555542, 1.1781134605407715, 1.18048095703125, 0.00012339458044152707, 0.3506201505661011, 0.23163697123527527, 0.016799096018075943, 4.070028480906984e-10, 2.5629777908325195, 1.3559610843658447], "max_p": 0.6785377264022827, "max_p_per_token": [0.6634165048599243, 0.36842212080955505, 0.4496222734451294, 0.44999679923057556, 0.5164543390274048, 0.7752082943916321, 0.9822284579277039, 0.9986577033996582, 0.8972044587135315, 0.21741995215415955, 0.6676487922668457, 0.44366469979286194, 0.4861195981502533, 0.9999901056289673, 0.9162480235099792, 0.9572855234146118, 0.9980985522270203, 1.0, 0.23762141168117523, 0.5454467535018921], "n_positions_probed": 1, "per_restart_best": [2.906998872756958]}
+{"step": 460, "discrete_loss": 3.9320104122161865, "best_sample_loss": 2.917154550552368, "soft_loss": 1.7836681604385376, "best_discrete": 2.906998872756958, "best_soft": 1.7836681604385376, "best_argmax": 3.9320104122161865, "best_sampling": 2.906998872756958, "relax_gap": 0.5463724727439838, "n_match": 7, "g_first_norm": 134.9879150390625, "vocab_size": 50257, "entropy": 0.8520523309707642, "entropy_per_token": [1.180050015449524, 1.1492443084716797, 1.1869146823883057, 1.0680891275405884, 0.7026029229164124, 0.6285346746444702, 0.09186773747205734, 0.010928496718406677, 0.322568416595459, 3.036736488342285, 0.6983737945556641, 1.2128170728683472, 1.1898845434188843, 0.00012256953050382435, 0.36196455359458923, 0.2752659022808075, 0.01775394007563591, 3.909309542748929e-10, 2.553407669067383, 1.3539202213287354], "max_p": 0.6758098006248474, "max_p_per_token": [0.6375701427459717, 0.3536594808101654, 0.43624329566955566, 0.4504512548446655, 0.5181794166564941, 0.7800112962722778, 0.9815966486930847, 0.9987010955810547, 0.9015963077545166, 0.22981519997119904, 0.6719741821289062, 0.4294532835483551, 0.4834930896759033, 0.9999902248382568, 0.9126514196395874, 0.9467088580131531, 0.9979789853096008, 1.0, 0.23736107349395752, 0.5487605929374695], "n_positions_probed": 1, "per_restart_best": [2.906998872756958]}
+{"step": 461, "discrete_loss": 3.898554563522339, "best_sample_loss": 3.0351414680480957, "soft_loss": 1.758710265159607, "best_discrete": 2.906998872756958, "best_soft": 1.758710265159607, "best_argmax": 3.898554563522339, "best_sampling": 2.906998872756958, "relax_gap": 0.548881454266318, "n_match": 8, "g_first_norm": 137.56893920898438, "vocab_size": 50257, "entropy": 0.8559291958808899, "entropy_per_token": [1.1678954362869263, 1.1482470035552979, 1.1834056377410889, 1.0655763149261475, 0.7022731304168701, 0.623600959777832, 0.09432848542928696, 0.010607494041323662, 0.31443899869918823, 3.0466973781585693, 0.6997749209403992, 1.2473284006118774, 1.1980758905410767, 0.00012160977348685265, 0.3730340003967285, 0.32533156871795654, 0.018743030726909637, 3.755445121544909e-10, 2.549987554550171, 1.3491159677505493], "max_p": 0.6764237284660339, "max_p_per_token": [0.6434980034828186, 0.36955273151397705, 0.448946088552475, 0.4496646821498871, 0.5168842673301697, 0.7830398678779602, 0.9809756278991699, 0.9987437129020691, 0.9052416086196899, 0.22180138528347015, 0.6722038388252258, 0.42689570784568787, 0.48174166679382324, 0.9999903440475464, 0.9090856909751892, 0.9337372779846191, 0.997853696346283, 1.0, 0.23551787436008453, 0.5531005859375], "n_positions_probed": 1, "per_restart_best": [2.906998872756958]}
+{"step": 462, "discrete_loss": 3.898554563522339, "best_sample_loss": 2.956368923187256, "soft_loss": 1.7320034503936768, "best_discrete": 2.906998872756958, "best_soft": 1.7320034503936768, "best_argmax": 3.898554563522339, "best_sampling": 2.906998872756958, "relax_gap": 0.5557318944309416, "n_match": 8, "g_first_norm": 134.0581817626953, "vocab_size": 50257, "entropy": 0.8602261543273926, "entropy_per_token": [1.1560250520706177, 1.1467078924179077, 1.200597882270813, 1.0632930994033813, 0.701960563659668, 0.618236780166626, 0.09685250371694565, 0.010282938368618488, 0.30664607882499695, 3.0344982147216797, 0.6991908550262451, 1.2819126844406128, 1.2080802917480469, 0.00012109423551009968, 0.38346731662750244, 0.3827277719974518, 0.0197906531393528, 3.6009109583012844e-10, 2.543834924697876, 1.3502962589263916], "max_p": 0.6760153770446777, "max_p_per_token": [0.6493203639984131, 0.38553890585899353, 0.44072848558425903, 0.4479076564311981, 0.5158452987670898, 0.786311149597168, 0.9803332686424255, 0.9987865090370178, 0.9086748361587524, 0.22817426919937134, 0.6744365692138672, 0.41587138175964355, 0.47507229447364807, 0.9999903440475464, 0.905688464641571, 0.9177688360214233, 0.9977194666862488, 1.0, 0.2383565455675125, 0.5537822246551514], "n_positions_probed": 1, "per_restart_best": [2.906998872756958]}
+{"step": 463, "discrete_loss": 3.898554563522339, "best_sample_loss": 2.9816982746124268, "soft_loss": 1.7052885293960571, "best_discrete": 2.906998872756958, "best_soft": 1.7052885293960571, "best_argmax": 3.898554563522339, "best_sampling": 2.906998872756958, "relax_gap": 0.5625844138871482, "n_match": 8, "g_first_norm": 135.36399841308594, "vocab_size": 50257, "entropy": 0.8671207427978516, "entropy_per_token": [1.151960015296936, 1.1436491012573242, 1.1996252536773682, 1.0921299457550049, 0.7017418146133423, 0.6138818860054016, 0.09947134554386139, 0.009891999885439873, 0.29910823702812195, 3.0379767417907715, 0.7005302906036377, 1.31414794921875, 1.21693754196167, 0.00012030167272314429, 0.3935289680957794, 0.4474186897277832, 0.020894423127174377, 3.4493369271970664e-10, 2.549180269241333, 1.3502185344696045], "max_p": 0.6754961609840393, "max_p_per_token": [0.6514568328857422, 0.40187782049179077, 0.44902411103248596, 0.44459304213523865, 0.513325035572052, 0.7888891100883484, 0.9796608686447144, 0.9988380074501038, 0.9119417071342468, 0.22308817505836487, 0.6744990944862366, 0.4156115651130676, 0.46810927987098694, 0.9999904632568359, 0.9023698568344116, 0.8983290195465088, 0.9975767731666565, 1.0, 0.2358974814414978, 0.5548443794250488], "n_positions_probed": 1, "per_restart_best": [2.906998872756958]}
+{"step": 464, "discrete_loss": 3.898554563522339, "best_sample_loss": 2.986149549484253, "soft_loss": 1.6777207851409912, "best_discrete": 2.906998872756958, "best_soft": 1.6777207851409912, "best_argmax": 3.898554563522339, "best_sampling": 2.906998872756958, "relax_gap": 0.5696556870490039, "n_match": 8, "g_first_norm": 133.6716766357422, "vocab_size": 50257, "entropy": 0.8719725012779236, "entropy_per_token": [1.1442296504974365, 1.1405911445617676, 1.2079219818115234, 1.089849591255188, 0.701826810836792, 0.6090885400772095, 0.10210666060447693, 0.009540073573589325, 0.2917902171611786, 3.0239436626434326, 0.7010624408721924, 1.3475496768951416, 1.226100206375122, 0.00011999922571703792, 0.4029572606086731, 0.5191475749015808, 0.02205086499452591, 3.3046232417177634e-10, 2.5474953651428223, 1.3520784378051758], "max_p": 0.6741310954093933, "max_p_per_token": [0.6552190184593201, 0.41639992594718933, 0.44330546259880066, 0.442855566740036, 0.5108978748321533, 0.7916784286499023, 0.9789783358573914, 0.9988841414451599, 0.915060818195343, 0.22866982221603394, 0.6753190159797668, 0.4020323157310486, 0.4589603841304779, 0.9999904632568359, 0.8992288112640381, 0.8749220371246338, 0.997425377368927, 1.0, 0.23828467726707458, 0.5545094013214111], "n_positions_probed": 1, "per_restart_best": [2.906998872756958]}
+{"step": 465, "discrete_loss": 3.898554563522339, "best_sample_loss": 2.9882078170776367, "soft_loss": 1.649595022201538, "best_discrete": 2.906998872756958, "best_soft": 1.649595022201538, "best_argmax": 3.898554563522339, "best_sampling": 2.906998872756958, "relax_gap": 0.5768700949740893, "n_match": 8, "g_first_norm": 135.46189880371094, "vocab_size": 50257, "entropy": 0.8570125699043274, "entropy_per_token": [1.1407002210617065, 1.1368855237960815, 1.2080858945846558, 1.088274359703064, 0.7015924453735352, 0.18786819279193878, 0.10497093200683594, 0.009089786559343338, 0.2840781807899475, 3.021557092666626, 0.7028936743736267, 1.3770372867584229, 1.2332826852798462, 0.00011955937225138769, 0.4117588400840759, 0.5974019765853882, 0.02327776327729225, 3.1614727502571327e-10, 2.559009075164795, 1.3523681163787842], "max_p": 0.6812916994094849, "max_p_per_token": [0.6570426225662231, 0.43050557374954224, 0.45015770196914673, 0.4434574544429779, 0.50859534740448, 0.9567952752113342, 0.9782297611236572, 0.998943030834198, 0.918294370174408, 0.22551500797271729, 0.6745925545692444, 0.4038423001766205, 0.45071178674697876, 0.9999904632568359, 0.8962640166282654, 0.8470091819763184, 0.9972631931304932, 1.0, 0.23378406465053558, 0.5548391938209534], "n_positions_probed": 1, "per_restart_best": [2.906998872756958]}
+{"step": 466, "discrete_loss": 3.898554563522339, "best_sample_loss": 3.0489003658294678, "soft_loss": 1.6237049102783203, "best_discrete": 2.906998872756958, "best_soft": 1.6237049102783203, "best_argmax": 3.898554563522339, "best_sampling": 2.906998872756958, "relax_gap": 0.5835110465117346, "n_match": 8, "g_first_norm": 135.50527954101562, "vocab_size": 50257, "entropy": 0.8633098602294922, "entropy_per_token": [1.1376640796661377, 1.1337323188781738, 1.2158188819885254, 1.0898188352584839, 0.7012380361557007, 0.19114099442958832, 0.10738936066627502, 0.008774826303124428, 0.2801940441131592, 3.0053961277008057, 0.7031221389770508, 1.4121447801589966, 1.2430285215377808, 0.00011970919877057895, 0.42057713866233826, 0.6820501089096069, 0.024422435089945793, 3.023216954556318e-10, 2.5518956184387207, 1.3576686382293701], "max_p": 0.6781048774719238, "max_p_per_token": [0.6584911346435547, 0.4417090117931366, 0.446125328540802, 0.43468064069747925, 0.5097589492797852, 0.9557830691337585, 0.9775922894477844, 0.9989839196205139, 0.9199151992797852, 0.2280806452035904, 0.6754872798919678, 0.37915530800819397, 0.43750303983688354, 0.9999904632568359, 0.8932654857635498, 0.813848614692688, 0.9971103668212891, 1.0, 0.24196097254753113, 0.5526555180549622], "n_positions_probed": 1, "per_restart_best": [2.906998872756958]}
+{"step": 467, "discrete_loss": 3.898554563522339, "best_sample_loss": 2.906998872756958, "soft_loss": 1.5924372673034668, "best_discrete": 2.906998872756958, "best_soft": 1.5924372673034668, "best_argmax": 3.898554563522339, "best_sampling": 2.906998872756958, "relax_gap": 0.5915313633921024, "n_match": 8, "g_first_norm": 138.1497039794922, "vocab_size": 50257, "entropy": 0.8735570907592773, "entropy_per_token": [1.129791498184204, 1.1308469772338867, 1.2158927917480469, 1.0904805660247803, 0.7007850408554077, 0.19253754615783691, 0.11072144657373428, 0.08380105346441269, 0.2731561064720154, 2.9900293350219727, 0.702838659286499, 1.4365684986114502, 1.2488627433776855, 0.00011949749023187906, 0.4280886948108673, 0.7711628675460815, 0.025722116231918335, 2.8838478827175607e-10, 2.5803160667419434, 1.359420895576477], "max_p": 0.6769367456436157, "max_p_per_token": [0.662286639213562, 0.45229393243789673, 0.45376625657081604, 0.4410358667373657, 0.5126352310180664, 0.9553508162498474, 0.9767060875892639, 0.9861667156219482, 0.9228001832962036, 0.23038628697395325, 0.6767517924308777, 0.395380437374115, 0.42878612875938416, 0.9999904632568359, 0.8906856179237366, 0.77513188123703, 0.9969348907470703, 1.0, 0.2291346788406372, 0.5525095462799072], "n_positions_probed": 1, "per_restart_best": [2.906998872756958]}
+{"step": 468, "discrete_loss": 3.898554563522339, "best_sample_loss": 2.9658544063568115, "soft_loss": 1.5626814365386963, "best_discrete": 2.906998872756958, "best_soft": 1.5626814365386963, "best_argmax": 3.898554563522339, "best_sampling": 2.906998872756958, "relax_gap": 0.5991638923922574, "n_match": 8, "g_first_norm": 134.0054168701172, "vocab_size": 50257, "entropy": 0.8792557120323181, "entropy_per_token": [1.1268514394760132, 1.1281064748764038, 1.2259259223937988, 1.0919550657272339, 0.700697124004364, 0.1948946714401245, 0.11311431229114532, 0.08050408959388733, 0.2703809440135956, 2.9736599922180176, 0.7055962681770325, 1.4729259014129639, 1.257198691368103, 0.00012017838162137195, 0.4357070028781891, 0.8601434230804443, 0.026941493153572083, 2.7539356906025603e-10, 2.557436943054199, 1.3629528284072876], "max_p": 0.6726273894309998, "max_p_per_token": [0.6634209156036377, 0.46198734641075134, 0.4469306766986847, 0.4279601275920868, 0.5091431140899658, 0.9546068906784058, 0.9760637283325195, 0.9868156313896179, 0.9244964718818665, 0.2334655076265335, 0.6751123666763306, 0.36451205611228943, 0.4151536822319031, 0.9999904632568359, 0.888052225112915, 0.731948971748352, 0.9967688322067261, 1.0, 0.24535273015499115, 0.5507656931877136], "n_positions_probed": 1, "per_restart_best": [2.906998872756958]}
+{"step": 469, "discrete_loss": 3.898554563522339, "best_sample_loss": 4.6541032791137695, "soft_loss": 1.5310941934585571, "best_discrete": 2.906998872756958, "best_soft": 1.5310941934585571, "best_argmax": 3.898554563522339, "best_sampling": 2.906998872756958, "relax_gap": 0.6072661884010633, "n_match": 8, "g_first_norm": 140.9064178466797, "vocab_size": 50257, "entropy": 0.8870240449905396, "entropy_per_token": [1.11733877658844, 1.1255592107772827, 1.2236802577972412, 1.0931774377822876, 0.7002706527709961, 0.1954096555709839, 0.11682053655385971, 0.07433297485113144, 0.2622639238834381, 2.9837796688079834, 0.7088097333908081, 1.492620587348938, 1.261383056640625, 0.00012013606465188786, 0.4419943690299988, 0.9477118253707886, 0.02838251367211342, 2.6263513586144427e-10, 2.6053073406219482, 1.3615176677703857], "max_p": 0.6702999472618103, "max_p_per_token": [0.6678719520568848, 0.47143203020095825, 0.45854416489601135, 0.4387243688106537, 0.5120193362236023, 0.9544472098350525, 0.975059449672699, 0.9880167245864868, 0.9277659058570862, 0.20163309574127197, 0.672934889793396, 0.38929978013038635, 0.4091557264328003, 0.9999904632568359, 0.8858560919761658, 0.6845573782920837, 0.9965705871582031, 1.0, 0.2199757993221283, 0.5521436929702759], "n_positions_probed": 1, "per_restart_best": [2.906998872756958]}
+{"step": 470, "discrete_loss": 3.5428948402404785, "best_sample_loss": 4.3931403160095215, "soft_loss": 1.5704925060272217, "best_discrete": 2.906998872756958, "best_soft": 1.5310941934585571, "best_argmax": 3.5428948402404785, "best_sampling": 2.906998872756958, "relax_gap": 0.5567205415781907, "n_match": 8, "g_first_norm": 143.62669372558594, "vocab_size": 50257, "entropy": 0.8895303606987, "entropy_per_token": [1.1358047723770142, 1.1242486238479614, 1.2312440872192383, 1.09696364402771, 0.7003432512283325, 0.197485089302063, 0.12126210331916809, 0.07216158509254456, 0.26070213317871094, 2.945082426071167, 0.7723343372344971, 1.526473045349121, 1.2673397064208984, 0.0001221998390974477, 0.4488636255264282, 1.0216363668441772, 0.029510360211133957, 2.478156013729915e-10, 2.4814863204956055, 1.3575435876846313], "max_p": 0.669733464717865, "max_p_per_token": [0.6584096550941467, 0.4804357588291168, 0.4566529095172882, 0.433825820684433, 0.5025709867477417, 0.9537932872772217, 0.9738412499427795, 0.9884328246116638, 0.9284347891807556, 0.23500734567642212, 0.6657757759094238, 0.3580508828163147, 0.3992324769496918, 0.9999902248382568, 0.8834284543991089, 0.6400539875030518, 0.9964136481285095, 1.0, 0.28713342547416687, 0.5531842708587646], "n_positions_probed": 1, "per_restart_best": [2.906998872756958]}
+{"step": 471, "discrete_loss": 3.898554563522339, "best_sample_loss": 3.038553476333618, "soft_loss": 1.5156946182250977, "best_discrete": 2.906998872756958, "best_soft": 1.5156946182250977, "best_argmax": 3.5428948402404785, "best_sampling": 2.906998872756958, "relax_gap": 0.6112162614300646, "n_match": 8, "g_first_norm": 156.4644775390625, "vocab_size": 50257, "entropy": 0.9009050726890564, "entropy_per_token": [1.124753713607788, 1.1263105869293213, 1.2359950542449951, 1.1002651453018188, 0.6997003555297852, 0.19734835624694824, 0.12663127481937408, 0.06741811335086823, 0.25131797790527344, 2.9522616863250732, 0.7739253044128418, 1.5414400100708008, 1.2700350284576416, 0.00012209788837935776, 0.4531269669532776, 1.080625295639038, 0.030957039445638657, 2.3762938838878256e-10, 2.626657247543335, 1.3592102527618408], "max_p": 0.6672049760818481, "max_p_per_token": [0.6632298827171326, 0.4850132167339325, 0.46065178513526917, 0.4399160146713257, 0.5116633176803589, 0.9538626670837402, 0.9723462462425232, 0.9893332123756409, 0.9321209788322449, 0.23050172626972198, 0.663933277130127, 0.39445656538009644, 0.40022432804107666, 0.9999902248382568, 0.8819131255149841, 0.6042388677597046, 0.996212363243103, 1.0, 0.21093955636024475, 0.553551197052002], "n_positions_probed": 1, "per_restart_best": [2.906998872756958]}
+{"step": 472, "discrete_loss": 4.163578987121582, "best_sample_loss": 3.617995500564575, "soft_loss": 1.4786040782928467, "best_discrete": 2.906998872756958, "best_soft": 1.4786040782928467, "best_argmax": 3.5428948402404785, "best_sampling": 2.906998872756958, "relax_gap": 0.6448718559522143, "n_match": 8, "g_first_norm": 135.85800170898438, "vocab_size": 50257, "entropy": 0.8633872866630554, "entropy_per_token": [1.1318892240524292, 1.124860405921936, 1.2513360977172852, 1.1014468669891357, 0.6997900009155273, 0.20102152228355408, 0.1284952461719513, 0.06688696891069412, 0.2499147653579712, 2.924638271331787, 0.7811065912246704, 1.5899922847747803, 0.5259276032447815, 0.00012403447180986404, 0.4584541916847229, 1.1167774200439453, 0.032081712037324905, 2.2646123865044387e-10, 2.5205297470092773, 1.3624749183654785], "max_p": 0.6888936161994934, "max_p_per_token": [0.6584822535514832, 0.49266013503074646, 0.44772353768348694, 0.43981996178627014, 0.5029252171516418, 0.9526878595352173, 0.9718217849731445, 0.98943030834198, 0.9327202439308167, 0.24345624446868896, 0.6602627635002136, 0.3285955786705017, 0.8706963062286377, 0.9999901056289673, 0.8800187706947327, 0.5853271484375, 0.9960536956787109, 1.0, 0.2737015187740326, 0.551499605178833], "n_positions_probed": 1, "per_restart_best": [2.906998872756958]}
+{"step": 473, "discrete_loss": 4.801446437835693, "best_sample_loss": 3.7586829662323, "soft_loss": 2.113800048828125, "best_discrete": 2.906998872756958, "best_soft": 1.4786040782928467, "best_argmax": 3.5428948402404785, "best_sampling": 2.906998872756958, "relax_gap": 0.5597576529915547, "n_match": 7, "g_first_norm": 611.229248046875, "vocab_size": 50257, "entropy": 0.8728126883506775, "entropy_per_token": [1.0472407341003418, 1.1197278499603271, 1.195203185081482, 1.1090803146362305, 0.6975604295730591, 0.1924329698085785, 0.13149607181549072, 0.05557765066623688, 0.24086014926433563, 2.8815176486968994, 0.7712621688842773, 1.6243867874145508, 0.7752305269241333, 0.00012806360609829426, 0.4739466905593872, 1.1645241975784302, 0.03544105961918831, 2.5516810886472285e-10, 2.610966682434082, 1.3296688795089722], "max_p": 0.6813299059867859, "max_p_per_token": [0.6975586414337158, 0.5093119144439697, 0.5260882377624512, 0.4360879063606262, 0.531018078327179, 0.9552249908447266, 0.9709712266921997, 0.9915125966072083, 0.9361541867256165, 0.25258904695510864, 0.6629629135131836, 0.3459990322589874, 0.7729385495185852, 0.9999897480010986, 0.8739712238311768, 0.5659213662147522, 0.9955869913101196, 1.0, 0.17472073435783386, 0.4279906451702118], "n_positions_probed": 1, "per_restart_best": [2.906998872756958]}
+{"step": 474, "discrete_loss": 4.942091464996338, "best_sample_loss": 4.2101874351501465, "soft_loss": 1.7067058086395264, "best_discrete": 2.906998872756958, "best_soft": 1.4786040782928467, "best_argmax": 3.5428948402404785, "best_sampling": 2.906998872756958, "relax_gap": 0.6546592023381763, "n_match": 8, "g_first_norm": 184.20770263671875, "vocab_size": 50257, "entropy": 0.8679046034812927, "entropy_per_token": [1.087774634361267, 1.1273400783538818, 1.253831148147583, 1.1149275302886963, 0.6989879608154297, 0.19463306665420532, 0.13183245062828064, 0.05754317343235016, 0.23743686079978943, 2.841514825820923, 0.7913182973861694, 1.6658252477645874, 0.7951128482818604, 0.000130146523588337, 0.46533915400505066, 1.13922119140625, 0.03699415922164917, 2.4135307641337533e-10, 2.35909366607666, 1.3592350482940674], "max_p": 0.6823204159736633, "max_p_per_token": [0.6781688332557678, 0.5063260793685913, 0.4733622074127197, 0.44841817021369934, 0.5143857598304749, 0.9544766545295715, 0.97087562084198, 0.9911496639251709, 0.9375013113021851, 0.2597365975379944, 0.648453950881958, 0.293987900018692, 0.7653210759162903, 0.9999896287918091, 0.8773146271705627, 0.5845572352409363, 0.9953635931015015, 1.0, 0.34678956866264343, 0.40022924542427063], "n_positions_probed": 1, "per_restart_best": [2.906998872756958]}
+{"step": 475, "discrete_loss": 4.478042125701904, "best_sample_loss": 3.7625558376312256, "soft_loss": 1.7445564270019531, "best_discrete": 2.906998872756958, "best_soft": 1.4786040782928467, "best_argmax": 3.5428948402404785, "best_sampling": 2.906998872756958, "relax_gap": 0.6104198267834505, "n_match": 8, "g_first_norm": 225.45697021484375, "vocab_size": 50257, "entropy": 0.8981063961982727, "entropy_per_token": [1.098125696182251, 1.1406079530715942, 1.2697958946228027, 1.1190168857574463, 0.6988874673843384, 0.19483047723770142, 0.13698676228523254, 0.052790869027376175, 0.22115936875343323, 2.8357229232788086, 0.8115999102592468, 1.6632143259048462, 0.8251103162765503, 0.00012781941040884703, 0.4500160217285156, 1.251330852508545, 0.03832492232322693, 2.2932213072923702e-10, 2.727569103240967, 1.4269115924835205], "max_p": 0.6757699847221375, "max_p_per_token": [0.6727931499481201, 0.4986783266067505, 0.46547871828079224, 0.4453579783439636, 0.506434977054596, 0.9544050693511963, 0.9693958759307861, 0.9920017719268799, 0.9434351325035095, 0.26374882459640503, 0.6334550976753235, 0.3859451711177826, 0.7514827847480774, 0.9999897480010986, 0.8829396367073059, 0.569107174873352, 0.9951756000518799, 1.0, 0.15769270062446594, 0.42788180708885193], "n_positions_probed": 1, "per_restart_best": [2.906998872756958]}
+{"step": 476, "discrete_loss": 4.057734966278076, "best_sample_loss": 4.324679851531982, "soft_loss": 1.672921895980835, "best_discrete": 2.906998872756958, "best_soft": 1.4786040782928467, "best_argmax": 3.5428948402404785, "best_sampling": 2.906998872756958, "relax_gap": 0.587720265151938, "n_match": 8, "g_first_norm": 191.82778930664062, "vocab_size": 50257, "entropy": 0.8864569067955017, "entropy_per_token": [1.1333369016647339, 1.140354871749878, 1.275351881980896, 1.117028832435608, 0.6972352266311646, 0.2001095712184906, 0.13718774914741516, 0.054807912558317184, 0.22368502616882324, 2.7801599502563477, 0.8211598992347717, 1.731810450553894, 0.8441051244735718, 0.00012944776972290128, 0.44804269075393677, 1.2449562549591064, 0.039720676839351654, 2.1877824551985725e-10, 2.407522678375244, 1.4324336051940918], "max_p": 0.6834433078765869, "max_p_per_token": [0.6543264985084534, 0.5050056576728821, 0.4659956097602844, 0.46157577633857727, 0.5282171964645386, 0.9526866674423218, 0.969338059425354, 0.9916378855705261, 0.942617654800415, 0.27583321928977966, 0.6305389404296875, 0.2836335599422455, 0.7451193928718567, 0.9999896287918091, 0.8837250471115112, 0.5743181705474854, 0.9949811100959778, 1.0, 0.3552038371562958, 0.45412248373031616], "n_positions_probed": 1, "per_restart_best": [2.906998872756958]}
+{"step": 477, "discrete_loss": 4.214943885803223, "best_sample_loss": 3.6874663829803467, "soft_loss": 1.6033966541290283, "best_discrete": 2.906998872756958, "best_soft": 1.4786040782928467, "best_argmax": 3.5428948402404785, "best_sampling": 2.906998872756958, "relax_gap": 0.6195924079725972, "n_match": 8, "g_first_norm": 203.31033325195312, "vocab_size": 50257, "entropy": 0.9038955569267273, "entropy_per_token": [1.1201353073120117, 1.1467797756195068, 1.2895554304122925, 1.1343631744384766, 0.6983728408813477, 0.20170371234416962, 0.14225473999977112, 0.051314253360033035, 0.20783445239067078, 2.7307963371276855, 0.8355327844619751, 1.7491999864578247, 0.8719370365142822, 0.00012818128743674606, 0.4395790100097656, 1.2366786003112793, 0.041442275047302246, 2.086941730539138e-10, 2.7069458961486816, 1.473357915878296], "max_p": 0.6781086325645447, "max_p_per_token": [0.66007399559021, 0.5046146512031555, 0.46376267075538635, 0.4226367771625519, 0.5053505301475525, 0.952184796333313, 0.9678617715835571, 0.9922593235969543, 0.9482069611549377, 0.29658326506614685, 0.6297496557235718, 0.3503608703613281, 0.7325543165206909, 0.9999897480010986, 0.8867884278297424, 0.578481137752533, 0.9947324991226196, 1.0, 0.20862340927124023, 0.46735796332359314], "n_positions_probed": 1, "per_restart_best": [2.906998872756958]}
+{"step": 478, "discrete_loss": 5.084344387054443, "best_sample_loss": 3.2200982570648193, "soft_loss": 1.4648563861846924, "best_discrete": 2.906998872756958, "best_soft": 1.4648563861846924, "best_argmax": 3.5428948402404785, "best_sampling": 2.906998872756958, "relax_gap": 0.7118888346913612, "n_match": 8, "g_first_norm": 164.92987060546875, "vocab_size": 50257, "entropy": 0.8974273800849915, "entropy_per_token": [1.1458344459533691, 1.1399638652801514, 1.3019800186157227, 1.1347671747207642, 0.6974720358848572, 0.20566362142562866, 0.141534224152565, 0.05116748809814453, 0.20752473175525665, 2.733454465866089, 0.8440577983856201, 1.8157354593276978, 0.9082119464874268, 0.00012872563092969358, 0.44005924463272095, 1.2556912899017334, 0.04318933188915253, 2.0186716187531317e-10, 2.407135009765625, 1.4749757051467896], "max_p": 0.6783393025398254, "max_p_per_token": [0.6456481218338013, 0.5171522498130798, 0.4597683250904083, 0.451895534992218, 0.5198025703430176, 0.9508745074272156, 0.968072772026062, 0.9922856092453003, 0.9483857154846191, 0.28088435530662537, 0.6313502192497253, 0.2644164264202118, 0.7169104814529419, 0.9999897480010986, 0.8867107033729553, 0.5715982913970947, 0.9944759011268616, 1.0, 0.29554101824760437, 0.47102317214012146], "n_positions_probed": 1, "per_restart_best": [2.906998872756958]}
+{"step": 479, "discrete_loss": 4.478042125701904, "best_sample_loss": 4.054871082305908, "soft_loss": 1.9252440929412842, "best_discrete": 2.906998872756958, "best_soft": 1.4648563861846924, "best_argmax": 3.5428948402404785, "best_sampling": 2.906998872756958, "relax_gap": 0.5700701246441457, "n_match": 8, "g_first_norm": 270.44879150390625, "vocab_size": 50257, "entropy": 0.8898243308067322, "entropy_per_token": [1.1650536060333252, 1.1645622253417969, 1.288852572441101, 1.1649633646011353, 0.6978230476379395, 0.2060401886701584, 0.14498326182365417, 0.0477839931845665, 0.19452765583992004, 2.615473508834839, 0.859791100025177, 1.8049752712249756, 0.9364160895347595, 0.00013190554454922676, 0.4275893568992615, 1.2278767824172974, 0.046535152941942215, 1.8587240080414347e-10, 2.3199777603149414, 1.4831290245056152], "max_p": 0.6881943941116333, "max_p_per_token": [0.6368146538734436, 0.5065990090370178, 0.49103665351867676, 0.3999846279621124, 0.5063468813896179, 0.9506834149360657, 0.9670586585998535, 0.9928818941116333, 0.9528502821922302, 0.33245599269866943, 0.6337518095970154, 0.33751118183135986, 0.7049621939659119, 0.99998939037323, 0.8911345601081848, 0.5855236649513245, 0.9939793348312378, 1.0, 0.4049873352050781, 0.4753356873989105], "n_positions_probed": 1, "per_restart_best": [2.906998872756958]}
+{"step": 480, "discrete_loss": 4.057734966278076, "best_sample_loss": 2.89390230178833, "soft_loss": 1.5388075113296509, "best_discrete": 2.89390230178833, "best_soft": 1.4648563861846924, "best_argmax": 3.5428948402404785, "best_sampling": 2.89390230178833, "relax_gap": 0.6207718039453155, "n_match": 8, "g_first_norm": 214.72689819335938, "vocab_size": 50257, "entropy": 0.9224128723144531, "entropy_per_token": [1.171055793762207, 1.1596287488937378, 1.348611831665039, 1.1767542362213135, 0.6977384090423584, 0.20739763975143433, 0.1433814913034439, 0.04857267811894417, 0.19282281398773193, 2.7478036880493164, 0.9049505591392517, 1.8869953155517578, 0.9874222278594971, 0.00013242423301562667, 0.4265490174293518, 1.2530665397644043, 0.048272229731082916, 1.8351450914444456e-10, 2.527540683746338, 1.519561529159546], "max_p": 0.6718980669975281, "max_p_per_token": [0.6359211802482605, 0.5146265029907227, 0.4350983798503876, 0.4171418249607086, 0.5069601535797119, 0.9501948356628418, 0.9675298929214478, 0.9927427768707275, 0.9535267949104309, 0.25656622648239136, 0.6109070777893066, 0.270929753780365, 0.6811445355415344, 0.99998939037323, 0.8915664553642273, 0.5771405696868896, 0.9937220215797424, 1.0, 0.30974939465522766, 0.47250330448150635], "n_positions_probed": 1, "per_restart_best": [2.89390230178833]}
+{"step": 481, "discrete_loss": 4.163578987121582, "best_sample_loss": 2.990140914916992, "soft_loss": 1.4276090860366821, "best_discrete": 2.89390230178833, "best_soft": 1.4276090860366821, "best_argmax": 3.5428948402404785, "best_sampling": 2.89390230178833, "relax_gap": 0.6571197302963538, "n_match": 8, "g_first_norm": 175.85533142089844, "vocab_size": 50257, "entropy": 0.9275345802307129, "entropy_per_token": [1.1615822315216064, 1.1914496421813965, 1.3263944387435913, 1.1815041303634644, 0.6976644396781921, 0.20713719725608826, 0.14837779104709625, 0.0427960604429245, 0.18148398399353027, 2.6515870094299316, 0.8858314156532288, 1.9058523178100586, 1.038536787033081, 0.00013391334505286068, 0.4310629069805145, 1.2779959440231323, 0.051341310143470764, 1.795065207588209e-10, 2.6214053630828857, 1.54855477809906], "max_p": 0.6735131144523621, "max_p_per_token": [0.6400507092475891, 0.52591472864151, 0.47885945439338684, 0.4114348590373993, 0.5012911558151245, 0.950247585773468, 0.9660494327545166, 0.9937456846237183, 0.957253098487854, 0.3086378574371338, 0.6339129209518433, 0.3179539740085602, 0.6531422734260559, 0.9999892711639404, 0.8900139331817627, 0.5643089413642883, 0.9932569265365601, 1.0, 0.2503717243671417, 0.4338268041610718], "n_positions_probed": 1, "per_restart_best": [2.89390230178833]}
+{"step": 482, "discrete_loss": 4.057734966278076, "best_sample_loss": 2.972071409225464, "soft_loss": 1.3645737171173096, "best_discrete": 2.89390230178833, "best_soft": 1.3645737171173096, "best_argmax": 3.5428948402404785, "best_sampling": 2.89390230178833, "relax_gap": 0.6637104866489214, "n_match": 8, "g_first_norm": 141.00462341308594, "vocab_size": 50257, "entropy": 0.9420837759971619, "entropy_per_token": [1.1754766702651978, 1.1771854162216187, 1.4044673442840576, 1.1857099533081055, 0.6975464224815369, 0.21076086163520813, 0.14743450284004211, 0.04183628410100937, 0.18081966042518616, 2.70048451423645, 0.9138671159744263, 1.9762041568756104, 1.0735766887664795, 0.00013566880079451948, 0.43299174308776855, 1.2860584259033203, 0.05363262817263603, 1.7513443473227142e-10, 2.637821674346924, 1.5456643104553223], "max_p": 0.6659315228462219, "max_p_per_token": [0.6303989887237549, 0.5389035940170288, 0.4303453266620636, 0.42883017659187317, 0.5025168657302856, 0.9490267634391785, 0.9663300514221191, 0.9939106702804565, 0.9575450420379639, 0.28043726086616516, 0.6202573776245117, 0.2469779998064041, 0.6360798478126526, 0.9999891519546509, 0.8894568681716919, 0.5665703415870667, 0.9929059743881226, 1.0, 0.24412404000759125, 0.44402357935905457], "n_positions_probed": 1, "per_restart_best": [2.89390230178833]}
+{"step": 483, "discrete_loss": 4.163578987121582, "best_sample_loss": 2.933906316757202, "soft_loss": 1.3154780864715576, "best_discrete": 2.89390230178833, "best_soft": 1.3154780864715576, "best_argmax": 3.5428948402404785, "best_sampling": 2.89390230178833, "relax_gap": 0.6840511275178208, "n_match": 8, "g_first_norm": 150.17636108398438, "vocab_size": 50257, "entropy": 0.9373572468757629, "entropy_per_token": [1.1943233013153076, 1.1722424030303955, 1.392120361328125, 1.1248199939727783, 0.6973531246185303, 0.21203553676605225, 0.14988885819911957, 0.03586355596780777, 0.17419582605361938, 2.629049777984619, 0.9076372981071472, 1.9733015298843384, 1.098146677017212, 0.0001373220729874447, 0.43585115671157837, 1.293367862701416, 0.056622449308633804, 1.6663853630305425e-10, 2.6533203125, 1.5468673706054688], "max_p": 0.6731416583061218, "max_p_per_token": [0.6192365288734436, 0.5483385324478149, 0.46015363931655884, 0.4691932797431946, 0.5042138695716858, 0.9485726952552795, 0.9655976295471191, 0.9949166774749756, 0.9596968293190002, 0.30949026346206665, 0.6327350735664368, 0.30858591198921204, 0.6220356225967407, 0.9999889135360718, 0.8885520100593567, 0.5621820688247681, 0.9924381375312805, 1.0, 0.2350960224866867, 0.44180893898010254], "n_positions_probed": 1, "per_restart_best": [2.89390230178833]}
+{"step": 484, "discrete_loss": 4.163578987121582, "best_sample_loss": 2.9839932918548584, "soft_loss": 1.3196513652801514, "best_discrete": 2.89390230178833, "best_soft": 1.3154780864715576, "best_argmax": 3.5428948402404785, "best_sampling": 2.89390230178833, "relax_gap": 0.6830487978342715, "n_match": 8, "g_first_norm": 164.82591247558594, "vocab_size": 50257, "entropy": 0.953557014465332, "entropy_per_token": [1.197332739830017, 1.1717840433120728, 1.4504873752593994, 1.159048318862915, 0.6972447633743286, 0.21649713814258575, 0.14883512258529663, 0.03496241569519043, 0.17558687925338745, 2.694441795349121, 0.9411509037017822, 2.044226884841919, 1.134739875793457, 0.0001387994270771742, 0.43826258182525635, 1.303389310836792, 0.05914326757192612, 1.6207396535961038e-10, 2.6584486961364746, 1.5454206466674805], "max_p": 0.6613025665283203, "max_p_per_token": [0.6143335700035095, 0.5522160530090332, 0.4073760211467743, 0.4335711598396301, 0.5010530948638916, 0.947060227394104, 0.9659125804901123, 0.9950662851333618, 0.9593686461448669, 0.2706563472747803, 0.6140457391738892, 0.24117447435855865, 0.6030686497688293, 0.9999887943267822, 0.8878014087677002, 0.562335193157196, 0.9920443296432495, 1.0, 0.23303557932376862, 0.445942759513855], "n_positions_probed": 1, "per_restart_best": [2.89390230178833]}
+{"step": 485, "discrete_loss": 4.163578987121582, "best_sample_loss": 2.959059000015259, "soft_loss": 1.2828514575958252, "best_discrete": 2.89390230178833, "best_soft": 1.2828514575958252, "best_argmax": 3.5428948402404785, "best_sampling": 2.89390230178833, "relax_gap": 0.6918873254083016, "n_match": 8, "g_first_norm": 153.6409149169922, "vocab_size": 50257, "entropy": 0.9467183947563171, "entropy_per_token": [1.208295464515686, 1.171661615371704, 1.4317728281021118, 1.1768665313720703, 0.697022557258606, 0.21811814606189728, 0.15260310471057892, 0.02972312644124031, 0.16930994391441345, 2.5908050537109375, 0.9160579442977905, 2.014807939529419, 1.1585355997085571, 0.00014061719411984086, 0.44096463918685913, 1.3017487525939941, 0.06241508573293686, 1.53447671236151e-10, 2.6486997604370117, 1.544817328453064], "max_p": 0.6704338192939758, "max_p_per_token": [0.6070988178253174, 0.5582011342048645, 0.4451983869075775, 0.44334766268730164, 0.5065262913703918, 0.9464929699897766, 0.9647819995880127, 0.9959176182746887, 0.9613894820213318, 0.31688612699508667, 0.6373420357704163, 0.3180040717124939, 0.5887232422828674, 0.9999886751174927, 0.8869264721870422, 0.560529887676239, 0.9915209412574768, 1.0, 0.23693984746932983, 0.44286036491394043], "n_positions_probed": 1, "per_restart_best": [2.89390230178833]}
+{"step": 486, "discrete_loss": 4.163578987121582, "best_sample_loss": 3.034048557281494, "soft_loss": 1.2803778648376465, "best_discrete": 2.89390230178833, "best_soft": 1.2803778648376465, "best_argmax": 3.5428948402404785, "best_sampling": 2.89390230178833, "relax_gap": 0.6924814279258303, "n_match": 8, "g_first_norm": 158.64404296875, "vocab_size": 50257, "entropy": 0.9653543829917908, "entropy_per_token": [1.2091768980026245, 1.1648526191711426, 1.489455223083496, 1.2034884691238403, 0.6970434784889221, 0.2229757010936737, 0.15156012773513794, 0.029321778565645218, 0.17135977745056152, 2.675058603286743, 0.9584846496582031, 2.094452381134033, 1.1993790864944458, 0.00014307358651421964, 0.4435591697692871, 1.3151350021362305, 0.06497633457183838, 1.4994674946144926e-10, 2.671924591064453, 1.5447402000427246], "max_p": 0.6573277711868286, "max_p_per_token": [0.6032764911651611, 0.5654622912406921, 0.3905004858970642, 0.4223911166191101, 0.4998716413974762, 0.9448148012161255, 0.9650959372520447, 0.9959821701049805, 0.9608739018440247, 0.27065446972846985, 0.6112846732139587, 0.24070009589195251, 0.5677413940429688, 0.9999884366989136, 0.8861024379730225, 0.5601001977920532, 0.9911126494407654, 1.0, 0.22257590293884277, 0.4480254352092743], "n_positions_probed": 1, "per_restart_best": [2.89390230178833]}
+{"step": 487, "discrete_loss": 4.163578987121582, "best_sample_loss": 2.9774415493011475, "soft_loss": 1.2308456897735596, "best_discrete": 2.89390230178833, "best_soft": 1.2308456897735596, "best_argmax": 3.5428948402404785, "best_sampling": 2.89390230178833, "relax_gap": 0.704377965788399, "n_match": 8, "g_first_norm": 151.55093383789062, "vocab_size": 50257, "entropy": 0.9551971554756165, "entropy_per_token": [1.22221839427948, 1.1608757972717285, 1.4710694551467896, 1.2173150777816772, 0.6968446969985962, 0.22472044825553894, 0.15545012056827545, 0.025045856833457947, 0.16507315635681152, 2.560474395751953, 0.9284482002258301, 2.064126491546631, 1.2223362922668457, 0.00014548443141393363, 0.44610726833343506, 1.3011245727539062, 0.06852764636278152, 1.4213061283463446e-10, 2.6295523643493652, 1.5444879531860352], "max_p": 0.6678240895271301, "max_p_per_token": [0.5944880843162537, 0.5722876787185669, 0.4245574474334717, 0.438245952129364, 0.5068078637123108, 0.944185733795166, 0.9639201760292053, 0.9966539144515991, 0.9628634452819824, 0.32221490144729614, 0.6373617053031921, 0.30677562952041626, 0.5546814203262329, 0.9999881982803345, 0.8852717876434326, 0.5635532140731812, 0.9905303120613098, 1.0, 0.24628497660160065, 0.4458085298538208], "n_positions_probed": 1, "per_restart_best": [2.89390230178833]}
+{"step": 488, "discrete_loss": 4.163578987121582, "best_sample_loss": 3.002012014389038, "soft_loss": 1.2297112941741943, "best_discrete": 2.89390230178833, "best_soft": 1.2297112941741943, "best_argmax": 3.5428948402404785, "best_sampling": 2.89390230178833, "relax_gap": 0.7046504226345099, "n_match": 8, "g_first_norm": 156.93466186523438, "vocab_size": 50257, "entropy": 0.9886810183525085, "entropy_per_token": [1.219624638557434, 1.1527044773101807, 1.5249685049057007, 1.243186116218567, 0.6968669295310974, 0.22943201661109924, 0.15461775660514832, 0.024375300854444504, 0.415662556886673, 2.660224199295044, 0.975228488445282, 2.1386525630950928, 1.2665985822677612, 0.0001485679968027398, 0.44907093048095703, 1.318737506866455, 0.07132697105407715, 1.390050713423463e-10, 2.684119939804077, 1.5480741262435913], "max_p": 0.6503743529319763, "max_p_per_token": [0.5929718017578125, 0.5796442627906799, 0.3708931505680084, 0.4121737480163574, 0.503505289554596, 0.9425204396247864, 0.9641724824905396, 0.9967579245567322, 0.903807520866394, 0.2692732512950897, 0.6069636344909668, 0.23707321286201477, 0.5328165888786316, 0.9999879598617554, 0.8843140006065369, 0.5604440569877625, 0.9900745749473572, 1.0, 0.21196675300598145, 0.4481249153614044], "n_positions_probed": 1, "per_restart_best": [2.89390230178833]}
+{"step": 489, "discrete_loss": 4.155917644500732, "best_sample_loss": 4.590950965881348, "soft_loss": 1.1850321292877197, "best_discrete": 2.89390230178833, "best_soft": 1.1850321292877197, "best_argmax": 3.5428948402404785, "best_sampling": 2.89390230178833, "relax_gap": 0.714856686138668, "n_match": 8, "g_first_norm": 155.8645782470703, "vocab_size": 50257, "entropy": 0.8770257830619812, "entropy_per_token": [1.2346631288528442, 1.1467682123184204, 1.5062588453292847, 1.2526021003723145, 0.6965985298156738, 0.23074761033058167, 0.15901240706443787, 0.020796317607164383, 0.40381765365600586, 0.588492751121521, 0.936186671257019, 2.0941221714019775, 1.2908650636672974, 0.00015123347111511976, 0.45166584849357605, 1.299037218093872, 0.07525153458118439, 1.3184758840267818e-10, 2.6048994064331055, 1.548579454421997], "max_p": 0.6910773515701294, "max_p_per_token": [0.5823614001274109, 0.5860480666160583, 0.3995158076286316, 0.43141499161720276, 0.5105916261672974, 0.9420262575149536, 0.962832510471344, 0.9973015189170837, 0.9076418280601501, 0.8924409747123718, 0.6386892795562744, 0.30800285935401917, 0.5202610492706299, 0.9999877214431763, 0.8834555149078369, 0.5652984976768494, 0.9894137382507324, 1.0, 0.25843167304992676, 0.4458308517932892], "n_positions_probed": 1, "per_restart_best": [2.89390230178833]}

claudini/methods/claude_random/v20/optimizer.py

@@ -0,0 +1,363 @@
+"""
+Claude v20 optimizer: Entropic simplex optimization with bandit sculpting.
+
+Extends EGD with K parallel restarts and discrete reward shaping.
+Each step combines two signals:
+  1. First-order soft gradient through soft embeddings -> entropic update
+     on ALL positions (global signal)
+  2. Bandit sculpting: forward-only probe B tokens spread across P positions
+     (L2R cycling). Per-token discrete losses become z-score rewards to
+     directly shape the distribution at each probed position.
+
+With K restarts (num_starts > 1), maintains K independent distributions.
+Candidate evaluations are batched across all restarts for efficiency.
+"""
+
+import json
+import logging
+from pathlib import Path
+
+import torch
+import torch.nn.functional as F
+from torch import Tensor
+from transformers import PreTrainedModel, PreTrainedTokenizerBase
+
+from claudini.methods.original.egd import EGDOptimizer
+
+log = logging.getLogger("claudini")
+
+
+class ClaudeV20Optimizer(EGDOptimizer):
+    """Entropic simplex optimization with bandit sculpting and K restarts.
+
+    Extends EGD with multi-start support and discrete reward shaping
+    at probed positions.
+    """
+
+    method_name = "claude_v20"
+    is_soft = True
+
+    def __init__(
+        self,
+        model: PreTrainedModel,
+        tokenizer: PreTrainedTokenizerBase,
+        optim_length: int = 20,
+        num_samples: int = 16,
+        lr: float = 0.1,
+        init_sigma: float = 10.0,
+        topk_per_position: int = 128,
+        sculpt_lr: float = 1.0,
+        positions_per_step: int = 1,
+        candidate_source: str = "theta",  # "theta" or "uniform"
+        accept_argmax: bool = True,  # if False, never accept argmax as new best
+        num_starts: int = 4,
+        seed: int | None = None,
+        allow_non_ascii: bool = False,
+    ):
+        super().__init__(model, tokenizer, optim_length, lr=lr, seed=seed, allow_non_ascii=allow_non_ascii)
+        self.num_starts = num_starts
+        self.num_samples = num_samples
+        self.init_sigma = init_sigma
+        self.topk_per_position = topk_per_position
+        self.sculpt_lr = sculpt_lr
+        self.positions_per_step = positions_per_step
+        self.candidate_source = candidate_source
+        self.accept_argmax = accept_argmax
+
+        # Per-restart state (initialized in setup)
+        self.thetas: list[Tensor] = []
+        self._restart_best_discrete_loss: list[float] = []
+        self._restart_best_discrete_ids: list[Tensor | None] = []
+        self._restart_best_argmax_loss: list[float] = []
+        self._restart_best_sample_loss: list[float] = []
+        self._restart_argmax_wins: list[int] = []
+        self._restart_sample_wins: list[int] = []
+        self._restart_current_pos: list[int] = []
+
+        # Global best across all restarts
+        self._best_discrete_loss: float = float("inf")
+        self._best_discrete_ids: Tensor | None = None
+        self._best_soft_loss: float = float("inf")
+        self._best_argmax_loss: float = float("inf")
+        self._best_sample_loss: float = float("inf")
+        self._argmax_wins: int = 0
+        self._sample_wins: int = 0
+
+        self._diag_trace: list[dict] = []
+
+    def setup(self, prompt: str, target: str) -> None:
+        self._prepare_prompt(prompt, target)
+
+        K = self.num_starts
+        device = self.model.device
+        m = self.optim_length
+        d = self.vocab_size
+
+        self.thetas = []
+        self._restart_best_discrete_loss = [float("inf")] * K
+        self._restart_best_discrete_ids = [None] * K
+        self._restart_best_argmax_loss = [float("inf")] * K
+        self._restart_best_sample_loss = [float("inf")] * K
+        self._restart_argmax_wins = [0] * K
+        self._restart_sample_wins = [0] * K
+        # Stagger starting positions for diversity when P < m
+        self._restart_current_pos = [(k * m // K) % m for k in range(K)]
+
+        self._best_discrete_loss = float("inf")
+        self._best_discrete_ids = None
+        self._best_soft_loss = float("inf")
+        self._best_argmax_loss = float("inf")
+        self._best_sample_loss = float("inf")
+        self._argmax_wins = 0
+        self._sample_wins = 0
+        self._diag_trace = []
+
+        for _ in range(K):
+            logits = torch.randn(m, d, dtype=torch.float32, device=device) * self.init_sigma
+            if self.forbidden_mask is not None:
+                logits[:, self.forbidden_mask] = -float("inf")
+            self.thetas.append(F.softmax(logits, dim=-1))
+
+    def step(self, step_num: int) -> tuple[float, float | None, str]:
+        K = self.num_starts
+        B = self.num_samples
+        P = min(self.positions_per_step, self.optim_length)
+        eta = self.lr
+        m = self.optim_length
+
+        # ---- 1. Soft gradient for each restart (K fwd+bwd) ----
+        g_softs = []
+        soft_loss_vals = []
+        for k in range(K):
+            theta_param = self.thetas[k].clone().detach().requires_grad_(True)
+            soft_loss = self.compute_soft_loss(theta_param)
+            soft_loss.backward()
+            g_softs.append(theta_param.grad.clone())
+            soft_loss_vals.append(soft_loss.item())
+        self.flop_counter.count_forward_backward(self.total_seq_len, batch_size=K)
+
+        # ---- 2. Candidate generation per restart ----
+        samples_per_pos = B // P
+        B_actual = samples_per_pos * P
+
+        all_sampled = []
+        with torch.no_grad():
+            for k in range(K):
+                theta = self.thetas[k]
+                positions = [(self._restart_current_pos[k] + i) % m for i in range(P)]
+
+                if self._restart_best_discrete_ids[k] is not None:
+                    base_ids = self._restart_best_discrete_ids[k].clone()
+                else:
+                    base_ids = theta.argmax(dim=-1)
+
+                sampled = base_ids.unsqueeze(0).expand(B_actual, -1).clone()
+                k_top = min(self.topk_per_position, self.vocab_size)
+
+                if self.candidate_source == "theta":
+                    for i, p in enumerate(positions):
+                        start = i * samples_per_pos
+                        end = start + samples_per_pos
+                        _, topk_at_p = theta[p].topk(k_top)
+                        tok_choices = torch.randint(0, k_top, (samples_per_pos,), device=theta.device)
+                        sampled[start:end, p] = topk_at_p[tok_choices]
+                else:  # uniform
+                    if self.forbidden_mask is not None:
+                        allowed = (~self.forbidden_mask).nonzero(as_tuple=True)[0]
+                    else:
+                        allowed = torch.arange(self.vocab_size, device=theta.device)
+                    for i, p in enumerate(positions):
+                        start = i * samples_per_pos
+                        end = start + samples_per_pos
+                        tok_choices = allowed[torch.randint(0, len(allowed), (samples_per_pos,), device=theta.device)]
+                        sampled[start:end, p] = tok_choices
+
+                all_sampled.append(sampled)
+
+        # ---- 3. Batched discrete evaluation (K * B_actual candidates) ----
+        all_candidates = torch.cat(all_sampled, dim=0)  # [K*B_actual, m]
+        all_losses = self.compute_discrete_loss_batch(all_candidates)  # [K*B_actual]
+        self.flop_counter.count_forward(self.total_seq_len, batch_size=K * B_actual)
+
+        # ---- 4. Per-restart: entropic update + sculpting ----
+        per_restart_best_sample_loss = []
+        per_restart_best_sample_ids = []
+
+        with torch.no_grad():
+            for k in range(K):
+                theta = self.thetas[k]
+                losses_k = all_losses[k * B_actual : (k + 1) * B_actual]
+                sampled_k = all_sampled[k]
+                positions = [(self._restart_current_pos[k] + i) % m for i in range(P)]
+
+                # Best-of-B for this restart
+                best_idx = losses_k.argmin()
+                per_restart_best_sample_loss.append(losses_k[best_idx].item())
+                per_restart_best_sample_ids.append(sampled_k[best_idx])
+
+                # Entropic update
+                log_theta = torch.log(theta.clamp(min=1e-20))
+                log_theta_new = log_theta - eta * g_softs[k].detach()
+                if self.forbidden_mask is not None:
+                    log_theta_new[:, self.forbidden_mask] = -float("inf")
+                theta_new = F.softmax(log_theta_new, dim=-1)
+
+                # Bandit sculpting at each probed position
+                for i, p in enumerate(positions):
+                    start = i * samples_per_pos
+                    end = start + samples_per_pos
+                    pos_losses = losses_k[start:end].float()
+                    pos_tokens = sampled_k[start:end, p]
+
+                    if pos_losses.std() < 1e-8:
+                        continue  # all same loss -> no signal
+
+                    rewards = -(pos_losses - pos_losses.mean()) / pos_losses.std()
+
+                    reward_accum = torch.zeros(self.vocab_size, device=theta.device)
+                    count_accum = torch.zeros(self.vocab_size, device=theta.device)
+                    reward_accum.scatter_add_(0, pos_tokens, rewards)
+                    count_accum.scatter_add_(0, pos_tokens, torch.ones_like(rewards))
+
+                    tried_mask = count_accum > 0
+                    avg_reward = torch.zeros_like(reward_accum)
+                    avg_reward[tried_mask] = reward_accum[tried_mask] / count_accum[tried_mask]
+
+                    log_theta_pos = torch.log(theta_new[p].clamp(min=1e-20))
+                    log_theta_pos += self.sculpt_lr * avg_reward
+                    if self.forbidden_mask is not None:
+                        log_theta_pos[self.forbidden_mask] = -float("inf")
+                    theta_new[p] = F.softmax(log_theta_pos, dim=-1)
+
+                self.thetas[k] = theta_new
+
+        # ---- 5. Batched argmax evaluation (K projections) ----
+        with torch.no_grad():
+            argmax_ids = torch.stack([self.thetas[k].argmax(dim=-1) for k in range(K)])  # [K, m]
+        argmax_losses = self.compute_discrete_loss_batch(argmax_ids)  # [K]
+        self.flop_counter.count_forward(self.total_seq_len, batch_size=K)
+
+        # ---- 6. Per-restart best updates ----
+        for k in range(K):
+            disc_k = argmax_losses[k].item()
+            samp_k = per_restart_best_sample_loss[k]
+            samp_ids_k = per_restart_best_sample_ids[k]
+
+            # Track per-restart argmax/sample bests
+            if disc_k < self._restart_best_argmax_loss[k]:
+                self._restart_best_argmax_loss[k] = disc_k
+            if samp_k < self._restart_best_sample_loss[k]:
+                self._restart_best_sample_loss[k] = samp_k
+
+            # Argmax acceptance (per-restart)
+            if self.accept_argmax and disc_k < self._restart_best_discrete_loss[k]:
+                self._restart_best_discrete_loss[k] = disc_k
+                self._restart_best_discrete_ids[k] = argmax_ids[k].clone()
+                self._restart_argmax_wins[k] += 1
+
+            # Sampling acceptance (per-restart)
+            if samp_k < self._restart_best_discrete_loss[k]:
+                self._restart_best_discrete_loss[k] = samp_k
+                self._restart_best_discrete_ids[k] = samp_ids_k.clone()
+                self._restart_sample_wins[k] += 1
+
+        # Derive global bests from per-restart bests
+        best_k = min(range(K), key=lambda k: self._restart_best_discrete_loss[k])
+        self._best_discrete_loss = self._restart_best_discrete_loss[best_k]
+        self._best_discrete_ids = self._restart_best_discrete_ids[best_k]
+        self._best_argmax_loss = min(self._restart_best_argmax_loss)
+        self._best_sample_loss = min(self._restart_best_sample_loss)
+        self._argmax_wins = sum(self._restart_argmax_wins)
+        self._sample_wins = sum(self._restart_sample_wins)
+
+        best_soft = min(soft_loss_vals)
+        if best_soft < self._best_soft_loss:
+            self._best_soft_loss = best_soft
+
+        # ---- 7. Diagnostics (metrics from best restart) ----
+        with torch.no_grad():
+            theta_best = self.thetas[best_k]
+            entropy_per_token = -(theta_best * torch.log(theta_best.clamp(min=1e-20))).sum(-1)
+            entropy = entropy_per_token.mean().item()
+            max_p_per_token = theta_best.max(-1).values
+            max_p = max_p_per_token.mean().item()
+            g_soft_norm = g_softs[best_k].norm().item()
+
+            if self._best_discrete_ids is not None:
+                n_match = (argmax_ids[best_k] == self._best_discrete_ids).sum().item()
+            else:
+                n_match = 0
+
+            disc_best_k = argmax_losses[best_k].item()
+            relax_gap = (disc_best_k - best_soft) / max(disc_best_k, 1e-8)
+
+        self._diag_trace.append(
+            {
+                "step": step_num,
+                "discrete_loss": disc_best_k,
+                "best_sample_loss": min(per_restart_best_sample_loss),
+                "soft_loss": best_soft,
+                "best_discrete": self._best_discrete_loss,
+                "best_soft": self._best_soft_loss,
+                "best_argmax": self._best_argmax_loss,
+                "best_sampling": self._best_sample_loss,
+                "relax_gap": relax_gap,
+                "n_match": n_match,
+                "g_first_norm": g_soft_norm,
+                "vocab_size": self.vocab_size,
+                "entropy": entropy,
+                "entropy_per_token": entropy_per_token.tolist(),
+                "max_p": max_p,
+                "max_p_per_token": max_p_per_token.tolist(),
+                "n_positions_probed": P,
+                "per_restart_best": [self._restart_best_discrete_loss[kk] for kk in range(K)],
+            }
+        )
+
+        self.log("entropy", entropy, prog_bar=True)
+        self.log("match", n_match, prog_bar=True)
+
+        # ---- 8. Advance positions ----
+        for k in range(K):
+            self._restart_current_pos[k] = (self._restart_current_pos[k] + P) % m
+
+        # Report global best
+        report_loss = self._best_discrete_loss
+        report_ids = self._best_discrete_ids
+
+        optim_str = self.tokenizer.decode(report_ids)
+        self._step_ids = report_ids
+
+        return report_loss, None, optim_str
+
+    def run(self, prompt, target, num_steps=10000, **kwargs):
+        result = super().run(prompt, target, num_steps, **kwargs)
+        self._log_summary()
+        return result
+
+    def _log_summary(self):
+        K = self.num_starts
+        steps = len(self._diag_trace)
+        total = self._argmax_wins + self._sample_wins
+        per_r = " ".join(
+            f"R{k}={self._restart_best_discrete_loss[k]:.4f}"
+            f"(a:{self._restart_argmax_wins[k]}/s:{self._restart_sample_wins[k]})"
+            for k in range(K)
+        )
+        log.info(
+            f"[v20] {steps} steps, K={K}, {total} improvements | {per_r} | "
+            f"best_argmax={self._best_argmax_loss:.4f}, best_sampling={self._best_sample_loss:.4f}, "
+            f"best_overall={self._best_discrete_loss:.4f}"
+        )
+
+    def save_diagnostics(self, path: str | Path | None = None) -> None:
+        if not self._diag_trace:
+            return
+        if path is None:
+            path = Path(__file__).parent / "diagnostics.jsonl"
+        path = Path(path)
+        path.parent.mkdir(parents=True, exist_ok=True)
+        with open(path, "w") as f:
+            for entry in self._diag_trace:
+                f.write(json.dumps(entry) + "\n")
+        log.info(f"Saved {len(self._diag_trace)} diagnostic entries to {path}")

claudini/methods/claude_random/v21/__init__.py

@@ -0,0 +1,3 @@
+from .optimizer import ClaudeV21Optimizer
+
+__all__ = ["ClaudeV21Optimizer"]

claudini/methods/claude_random/v21/optimizer.py

@@ -0,0 +1,53 @@
+"""
+Claude v21 optimizer: Entropic simplex with tuned hyperparameters.
+
+v20 at defaults (lr=0.1, sculpt_lr=1.0) showed strong results.
+Testing lr=0.3 (3x default) + sculpt_lr=2.0 (2x default) for faster
+convergence and stronger discrete reward shaping.
+"""
+
+import logging
+
+from transformers import PreTrainedModel, PreTrainedTokenizerBase
+
+from claudini.methods.claude_random.v20.optimizer import ClaudeV20Optimizer
+
+logger = logging.getLogger("claudini")
+
+
+class ClaudeV21Optimizer(ClaudeV20Optimizer):
+    method_name = "claude_v21"
+
+    def __init__(
+        self,
+        model: PreTrainedModel,
+        tokenizer: PreTrainedTokenizerBase,
+        optim_length: int = 20,
+        num_samples: int = 16,
+        lr: float = 0.3,
+        init_sigma: float = 10.0,
+        topk_per_position: int = 128,
+        sculpt_lr: float = 2.0,
+        positions_per_step: int = 1,
+        candidate_source: str = "theta",
+        accept_argmax: bool = True,
+        num_starts: int = 1,
+        seed: int | None = None,
+        allow_non_ascii: bool = False,
+    ):
+        super().__init__(
+            model,
+            tokenizer,
+            optim_length,
+            num_samples,
+            lr,
+            init_sigma,
+            topk_per_position,
+            sculpt_lr,
+            positions_per_step,
+            candidate_source,
+            accept_argmax,
+            num_starts,
+            seed,
+            allow_non_ascii,
+        )

claudini/methods/claude_random/v22/__init__.py

@@ -0,0 +1,3 @@
+from .optimizer import ClaudeV22Optimizer
+
+__all__ = ["ClaudeV22Optimizer"]

claudini/methods/claude_random/v22/optimizer.py

@@ -0,0 +1,34 @@
+"""
+Claude v22 optimizer: ADC decoupled K/lr, K=16, lr=20, no LSGM.
+
+Same lr tuning approach as v16 (which halved loss on Qwen: 0.42→0.23).
+K=16 (ADC default) preserves step count. lr=20 (2× default) for faster convergence.
+No LSGM since it hurts on Llama-2.
+Targeting Llama-2 where ADC alone gets 5.33, claude_v20 gets 2.69.
+"""
+
+import logging
+
+from transformers import PreTrainedModel, PreTrainedTokenizerBase
+
+from claudini.methods.claude_random.v19 import ClaudeV19Optimizer
+
+logger = logging.getLogger("claudini")
+
+
+class ClaudeV22Optimizer(ClaudeV19Optimizer):
+    method_name = "claude_v22"
+
+    def __init__(
+        self,
+        model: PreTrainedModel,
+        tokenizer: PreTrainedTokenizerBase,
+        optim_length: int = 20,
+        lr: float = 20.0,
+        momentum: float = 0.99,
+        ema_alpha: float = 0.01,
+        num_starts: int = 16,
+        seed: int | None = None,
+        allow_non_ascii: bool = False,
+    ):
+        super().__init__(model, tokenizer, optim_length, lr, momentum, ema_alpha, num_starts, seed, allow_non_ascii)

claudini/methods/claude_random/v23/__init__.py

@@ -0,0 +1,3 @@
+from .optimizer import ClaudeV23Optimizer
+
+__all__ = ["ClaudeV23Optimizer"]

claudini/methods/claude_random/v23/optimizer.py

@@ -0,0 +1,53 @@
+"""
+Claude v23 optimizer: Entropic simplex with aggressive lr tuning.
+
+v21 (lr=0.3, sculpt_lr=2.0) showed promise.
+Pushing further: lr=0.5 (5x default), sculpt_lr=3.0 (3x default).
+K=1 (default) since K=4 hurt in v20 (fewer steps per restart).
+"""
+
+import logging
+
+from transformers import PreTrainedModel, PreTrainedTokenizerBase
+
+from claudini.methods.claude_random.v20.optimizer import ClaudeV20Optimizer
+
+logger = logging.getLogger("claudini")
+
+
+class ClaudeV23Optimizer(ClaudeV20Optimizer):
+    method_name = "claude_v23"
+
+    def __init__(
+        self,
+        model: PreTrainedModel,
+        tokenizer: PreTrainedTokenizerBase,
+        optim_length: int = 20,
+        num_samples: int = 16,
+        lr: float = 0.5,
+        init_sigma: float = 10.0,
+        topk_per_position: int = 128,
+        sculpt_lr: float = 3.0,
+        positions_per_step: int = 1,
+        candidate_source: str = "theta",
+        accept_argmax: bool = True,
+        num_starts: int = 1,
+        seed: int | None = None,
+        allow_non_ascii: bool = False,
+    ):
+        super().__init__(
+            model,
+            tokenizer,
+            optim_length,
+            num_samples,
+            lr,
+            init_sigma,
+            topk_per_position,
+            sculpt_lr,
+            positions_per_step,
+            candidate_source,
+            accept_argmax,
+            num_starts,
+            seed,
+            allow_non_ascii,
+        )

claudini/methods/claude_random/v24/__init__.py

@@ -0,0 +1,3 @@
+from .optimizer import ClaudeV24Optimizer
+
+__all__ = ["ClaudeV24Optimizer"]